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EVALUATING EFFORTS TO HELP FAMILIES 
SUPPORT THEIR CHILDREN AND ESCAPE 
POVERTY 


WEDNESDAY, JULY 17, 2013 

U.S. House of Representatives, 

Committee on Ways and Means, 
Subcommittee on Human Resources, 

Washington, DC. 

The Subcommittee met, pursuant to call, at 4:06 p.m., in room 
1100, Longworth House Office Building, the Honorable Dave 
Reichert [Chairman of the Subcommittee] presiding. 

[The advisory of the hearing follows:] 


( 1 ) 
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HEARING ADVISORY 

Chairman Reichert Announces Hearing on Evalu- 
ating Efforts to Help Families Support their 
Children and Escape Poverty 

1100 Longworth House Office Building at 4:00 PM 
Washington, July 10, 2013 

Congressman Dave Reichert (R-WA), Chairman of the Subcommittee on Human 
Resources of the Committee on Ways and Means, today announced that the Sub- 
committee will hold a hearing titled, “What Really Works: Evaluating Current Ef- 
forts to Help Families Support their Children and Escape Poverty.” The hearing will 
review evidence about the effectiveness of programs designed to assist low-income 
families and individuals, how Congress can ensure these programs are evaluated ef- 
fectively, and how funding can best be directed toward programs and services that 
have the greatest impact on reducing poverty. The hearing will take place at 
4:00 pm on Wednesday, July 17, 2013, in Room 1100 of the Longworth House 
Office Building. This hearing is the second in a three-part series of hearings on 
welfare reform issues. 

In view of the limited time available to hear from witnesses, oral testimony at 
this hearing will be from invited witnesses only. Witnesses will include experts on 
the evaluation of social programs, as well as experts who use high-quality evalua- 
tions to inform public policy decisions. However, any individual or organization not 
scheduled for an oral appearance may submit a written statement for consideration 
by the Committee and for inclusion in the printed record of the hearing. 

BACKGROUND : 

The Federal Government spends hundreds of billions each year on more than 80 
programs for families and individuals with low income. While each of these pro- 
grams is intended to alleviate poverty and improve the lives of those who receive 
these benefits, few programs have been rigorously evaluated to determine if they ac- 
tually achieve their goals. According to social policy experts writing about the eval- 
uation of Federal social programs in 2010, “[s]ince 1990, there have been 10 in- 
stances in which an entire Federal social program has been evaluated using the sci- 
entific ‘gold standard’ method” to determine whether the program really works, and 
“nine of these evaluations found weak or no positive effects.” 

Research has shown that dozens of specific interventions have demonstrated posi- 
tive results in addressing various social problems, including by reducing child mal- 
treatment, improving educational achievement, and increasing employment and 
earnings. However, in some cases, high-quality evaluations have revealed that some 
programs previously believed to be effective actually had no impact. In other cases, 
social programs expected to improve the lives of low-income adults or children actu- 
ally caused harm — meaning those who did not receive the service or benefit avoided 
the detrimental effects caused by the program because they did not participate. In 
addition, many Federal social programs have never been rigorously evaluated to de- 
termine whether they effectively address the problem they were created to solve, 
and evidence of effectiveness is not routinely used by Congress to address program 
deficiencies or redirect funding to more effective programs and policies. 

In announcing the hearing, Chairman Reichert stated, “Americans have always 
been willing to help those in need. But when the American people are 
asked to fund programs to help those most in need, they should be assured 
that their tax dollars are really making a positive difference. Unfortu- 
nately, few of our Nation’s social programs have been rigorously evaluated, 
and even fewer have shown that they are effective in addressing the prob- 
lems they set out to solve. It is critical that we learn more about what 
works to help low-income families, that we ensure these programs are eval- 
uated effectively, and that we focus taxpayer resources on those efforts 
that truly help families and children in need.” 
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FOCUS OF THE HEARING : 

The hearing will review what we know about the effectiveness of current pro- 
grams designed to assist low-income families and individuals, how Congress can en- 
sure more social programs are rigorously evaluated to determine their impact, and 
how high-quality evidence can best be used to inform the design of social programs 
at the federal level. 

DETATT,S FOR SUBMISSION OF WRITTEN COMMENTS: 

Please Note: Any person(s) and/or organization(s) wishing to submit for the hear- 
ing record must follow the appropriate link on the hearing page of the Committee 
website and complete the informational forms. From the Committee homepage, 
http:llwaysandmeans.house.gov, select “Hearings.” Select the hearing for which you 
would like to submit, and click on the link entitled, “Please click here to submit a 
statement or letter for the record.” Once you have followed the online instructions, 
submit all requested information. Attach your submission as a Word document, in 
compliance with the formatting requirements listed below, by Wednesday, July 
31, 2013. Finally, please note that due to the change in House mail policy, the U.S. 
Capitol Police will refuse sealed-package deliveries to all House Office Buildings. 
For questions, or if you encounter technical problems, please call ( 202 ) 225-1721 or 
( 202 ) 225 - 3625 . 

FORMATTING REQUIREMENTS : 

The Committee relies on electronic submissions for printing the official hearing 
record. As always, submissions will be included in the record according to the discre- 
tion of the Committee. The Committee will not alter the content of your submission, 
but we reserve the right to format it according to our guidelines. Any submission 
provided to the Committee by a witness, any supplementary materials submitted for 
the printed record, and any written comments in response to a request for written 
comments must conform to the guidelines listed below. Any submission or supple- 
mentary item not in compliance with these guidelines will not be printed, but will 
be maintained in the Committee files for review and use by the Committee. 

1. All submissions and supplementary materials must be provided in Word format and MUST 
NOT exceed a total of 10 pages, including attachments. Witnesses and submitters are advised 
that the Committee relies on electronic submissions for printing the official hearing record. 

2. Copies of whole documents submitted as exhibit material will not be accepted for printing. 
Instead, exhibit material should be referenced and quoted or paraphrased. All exhibit material 
not meeting these specifications will be maintained in the Committee files for review and use 
by the Committee. 

3. All submissions must include a list of all clients, persons, and/or organizations on whose 
behalf the witness appears. A supplemental sheet must accompany each submission listing the 
name, company, address, telephone, and fax numbers of each witness. 

The Committee seeks to make its facilities accessible to persons with disabilities. 
If you are in need of special accommodations, please call 202 - 225-1721 or 202 - 226 - 
3411 TTD/TTY in advance of the event (four business days notice is requested). 
Questions with regard to special accommodation needs in general (including avail- 
ability of Committee materials in alternative formats) may be directed to the Com- 
mittee as noted above. 

Note: All Committee advisories and news releases are available online at http:ll 
WWW. waysandmeans. house. gov L 


Chairman REICHERT. Good afternoon, the Committee will come 
to order. This is the second in our series of three hearings on wel- 
fare reform. In our first hearing, we learned that programs de- 
signed to help low-income families often don’t do enough to help re- 
cipients go to work and get ahead. Today we will explore what we 
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know about the effectiveness of such programs, how we can hold 
more programs accountable for their performance, and how we can 
ensure they provide real help so recipients can support their fami- 
lies and move up the economic ladder. Over one-third of American 
households receive low-income benefits today, and Federal spend- 
ing alone on these programs equals $15,000 per individual below 
the poverty line each year. Yet few programs can show that they 
improve outcomes for those in need. 

What we will hear today is that in many cases, these programs 
are either untested or have not been proven to work. According to 
program evaluation experts, “Since 1990, there have been 10 in- 
stances in which an entire Federal social program has been evalu- 
ated using the scientific gold standard method of randomly assign- 
ing individuals to a program or control group. Nine of the evalua- 
tions found weak or no positive effects.” 

In another example, a review of 13 rigorous studies on employ- 
ment and training programs showed three-quarters of them had 
weak or no positive effects on those that they were supposed to be 
helping. All of this comes at a cost. The programs in question con- 
tinued to spend literally billions of dollars every year without deliv- 
ering the results promised to those in need. 

We know many social programs lack meaningful outcomes, but 
some programs go further and can even be harmful. For example. 
Scared Straight — which I am familiar with as a former sheriff — or- 
ganized visits to prisons by juvenile delinquents with the goal of 
deterring them from future offending. However, instead of reducing 
crime, these programs actually increased the odds that participants 
will find themselves in trouble in the future. In fact, a comprehen- 
sive review of research by Washington State Institute for Public 
Policy estimated that every dollar spent on the program actually 
creates $76 in additional cost for taxpayers, crime victims, and the 
participants themselves because the youth who go through these 
programs are more likely to commit crimes in the future. 

This all suggests that more programs, including those in our ju- 
risdiction, should be evaluated to ensure the families are receiving 
real help. Ultimately, Congress and the administration should fund 
what works so we can deliver better results to those in need. This 
is an issue that can and should be bipartisan as it is all about 
doing right by recipients and taxpayers alike. 

Last week the Obama administration hosted a full-day con- 
ference on funding what works, highlighting how the private sector 
is willing to work with government to ensure that programs are 
really making a difference. Especially given our current fiscal cli- 
mate, it is important to ensure our resources are focused on efforts 
that have the greatest impact on those in need, and I am proud to 
say that my home State of Washington is a leader in this regard, 
as Steve Aos of the Washington State Institute for Public Policy 
will shortly tell us. We will also hear from experts from Utah and 
Texas, as well as national leaders, about what is being done and 
what more can be done to ensure that these programs are held ac- 
countable for producing real results. I look forward to all of your 
testimony today. 

Mr. Doggett, would you care to make an opening statement? 
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Mr. DOGGETT. Thank you very much, Mr. Chairman, and 
thanks to all of our witnesses. I welcome this opportunity to dis- 
cuss the programs and strategies that have proven to be most suc- 
cessful in helping our families escape poverty. Federal initiatives 
help raised 40 million Americans above the poverty line in 2011 
under a comprehensive measure that counts all assistance known 
as the supplemental poverty measure. 

Taken as a whole, public policy is having an immense impact on 
the well-being of many of our least fortunate neighbors. This, how- 
ever, still leaves the question of which specific approaches are most 
effective in achieving our objectives, and as we contemplate that 
question, I believe that the focus of this Subcommittee ought to be 
on the one program, the Temporary Assistance for Needy Families, 
that is within our jurisdiction. That ought to be our primary focus, 
especially since the TANF program is set to expire on September 
the 30th. We have very few legislative days prior to that time, with 
the Congress being out most of August and the beginning of Sep- 
tember, and I would suggest we get about the work on that specific 
piece of legislation. 

I voted for the 1996 welfare reform law myself because I believe 
that helping people find a job is the best strategy to reducing pov- 
erty. But this premise hinges on two very important principles. 
First, assistance has to be available when jobs are scarce, as they 
have been until very recently; and, second, a real effort has to be 
made to help people find, maintain, and advance in employment. 
Any fair reading of the last decade of the TANF program finds it 
lacking on both counts. The percentage of poor single mothers who 
are working has been dropping almost consistently for the past 12 
years, after having made significant progress in the mid and late 
nineties. 

Even more troubling, the percentage of poor mothers who are 
neither working nor receiving any assistance from TANF is more 
than twice as high as it was when TANF was established in 1996. 

Some of our colleagues often complain that our Federal programs 
are allowed to drift on autopilot. That seems to me to be accurate 
as it relates to TANF. This program is in real need of a significant 
reevaluation rather than this stop-start for brief periods approach 
that has been taken in recent years. Instead of working toward 
that goal, we spent most of the last year in this Subcommittee de- 
bating whether the administration was giving the States too much 
flexibility in the TANF program. 

For those who think that work requirements, stricter work re- 
quirements constitute a panacea on this issue, it is noteworthy that 
a number of States, including those that have Republican Gov- 
ernors, have complained that the current TANF work participation 
requirements really don’t measure success. Rather than continue 
the same tired old arguments, our Committee can actively advance 
the debate on this issue by reviewing evidence on specific strategies 
that might help TANF recipients get and retain jobs. One prom- 
ising approach is boosting both employment and earnings through 
sectoral training programs that target high-demand occupations 
and provide training and job search assistance to low-income indi- 
viduals. 
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Unlike some past training programs, these efforts are squarely 
aimed at preparing folks for job opportunities that exist in their 
communities. I look forward especially to having a native from Aus- 
tin, Tara Smith, with the Ray Marshall Center at the University 
of Texas offer comments about the success that is reflected in Cap- 
ital IDEA in Austin and Project QUEST in San Antonio that have 
shown real promise in helping people find not only jobs, but lasting 
careers. The Alamo Academies in San Antonio have taken this 
same successful sectoral employment approach and have partnered 
with high schools, community colleges, aerospace companies at Port 
San Antonio to provide specialized advanced manufacturing train- 
ing. 

I have been out to meet with some of those students. They are 
impressive. They are high school students who complete the pro- 
gram and receive valuable credentials along with their high school 
diploma when graduating, and some are averaging a starting pay 
of over $30,000 out of school each year. 

Mr. Chairman, I look forward to a productive discussion about 
how these and other proven strategies might help us to improve 
outcomes for TANF recipients and other struggling Americans. 
Let’s find a path forward toward our common goal of increasing 
employment and reducing poverty. Thank you very much. 

Chairman REICHERT. Thank you, Mr. Doggett. Without objec- 
tion, each Member will have the opportunity to submit a written 
statement and have it included in the record at this point. 

I want to remind our witnesses, please, to limit their oral state- 
ments to 5 minutes. However, without objection, all of the written 
testimony will be made a part of the permanent record. 

On our panel this afternoon we will be hearing from Jon Baron, 
president. Coalition for Evidence-Based Policy; &isten Cox, execu- 
tive director, Utah’s Governor’s Office of Management and Budget; 
Steve Aos, director of Washington State Institute for Public Policy; 
David Muhlhausen, Ph.D. research fellow. Empirical Policy Anal- 
ysis, The Heritage Foundation; and Tara Smith, research associate, 
Ray Marshall Center, Lyndon B. Johnson School of Public Affairs 
at The University of Texas. 

Welcome to all of you. Thank you for taking the time to be with 
us today. I will just let you know that you see three Members in 
front of you. Others are on the floor speaking on a bill, which I just 
came back from. That is why we started a little bit late. So we will 
have some other Members joining us here shortly. 

Mr. Baron, please proceed with your testimony. 

STATEMENT OF JON BARON, PRESIDENT, COALITION FOR 
EVIDENCE-BASED POLICY 

Mr. BARON. Thank you, Mr. Chairman. Chairman Reichert, 
Ranking Member Doggett, and Congressman Davis, I appreciate 
the opportunity to testify before the Subcommittee on behalf of the 
nonpartisan, nonprofit Coalition for Evidence-Based Policy. My tes- 
timony will address how evidence-based program reforms can 
greatly increase the effectiveness of government social spending in 
improving people’s lives. 

It is often assumed that the only way to increase government’s 
impact on social problems such as poverty and educational failure 
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is to spend more money, an assumption that conflicts with the cur- 
rent national interest in reducing the deficit. Largely overlooked, 
however, are clear examples from welfare and other areas where 
rigorous randomized trials, which as you mentioned, are widely 
considered the strongest method of evaluating program effective- 
ness, have identified program reforms that produced important im- 
provements in people’s lives while simultaneously reducing govern- 
ment spending. 

As an illustrative example in the eighties and nineties, govern- 
ment and foundations sponsored a large number of randomized 
trials of State and local welfare reforms. Three major reforms — two 
in California, one in Oregon — were found especially effective. They 
focused on moving welfare recipients quickly into the workforce 
through short-term job search, assistance, and training, and were 
found to produce gains in participants’ employment and earnings 
of 20 to 50 percent sustained over several years. 

Importantly, they also produced net savings to the government 
in reduced welfare and food stamps of between $1700 and $6,000 
per person. These findings helped build the political consensus for 
the strong work requirements in the 1996 Welfare Reform Act. 

A second example is in foster care where in the late nineties, 
HHS granted Illinois a waiver from Federal law to implement sub- 
sidized guardianship, which is an alternative to foster care in 
which the State pays a subsidy to the child’s relative or foster par- 
ent to serve as their subsidized guardian, as their legal guardian. 

Illinois evaluated subsidized guardianship in a large randomized 
trial which, over a 9-year period, found that the program increased 
children’s placement in a permanent home by 8 percent, reduced 
their days in foster care by 16 percent, and produced net savings 
to the foster care system of about $2300 per child. Based on those 
findings, CBO scored savings of $800 million for Federal legislation 
that was enacted in 2008 to expand subsidized guardianship na- 
tionally. To identify enough of these reforms to generate broad- 
based improvement in government effectiveness will require stra- 
tegic trial and error. In other words, rigorously testing many prom- 
ising reforms to identify the few that are effective. The instances 
of effectiveness that I just described are exceptions that have 
emerged from testing a much larger pool. 

More generally, most innovations, typically 80 to 90 percent, are 
found to produce weak or no positive effects when rigorously evalu- 
ated, a pattern that occurs not just in social spending, but in other 
fields where randomized trials are done, including medicine and 
business. 

In my testimony, I offer concrete suggestions for the Subcommit- 
tee’s consideration to greatly accelerate the rate of program innova- 
tion and rigorous testing in social spending so as to grow the num- 
ber of proven cost saving reforms like those I discussed. I suggest, 
for example, authorizing greater use of Federal waivers to stimu- 
late State and local innovation and evidence building, which was 
a tool deployed with great success in welfare reform under both Re- 
publican and Democratic administrations. I also suggest steps this 
Subcommittee can take to facilitate greater use of low cost random- 
ized control trials, such as the subsidized guardianship trial that 
I described earlier, which cost just $100,000 to conduct, yet identi- 
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fied an innovation that CBO scored as saving $800 million. These 
suggestions are designed to catalyze evidence-driven improvements 
in a social spending system that, in many cases, has fallen well 
short of its intended goals. 

The American poverty rate, for example, now at 15 percent, has 
shown little change, whether by official or the supplemental meas- 
ures, the National Academy measures since the seventies. In K-12 
education, reading and math achievement of 17-year-olds, who are 
the end product of our K-12 system, is virtually unchanged over 
the past 40 years according to official measures, even though there 
has been a 90 percent increase in public spending per student, ad- 
justed for inflation, since that time. Evidence-based policy offers a 
demonstrated path to more effective, less expensive government. 
Thank you. 

Chairman REICHERT. Thank you, Mr. Baron. 

[The prepared statement of Mr. Baron follows:] 
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House CominUtee on Ways anti Means. SubcotmnUtee on Human Resources 
Hearing on WbiU Works 'Evaluation, July 1 7. 201S 



Statement of Jon Baron 
President, Coalition for Evidence^ased Policy 


Chaiimaii Reicheit. Ranking Member Doggett, and Members of the Ways and Means Subcoinmitte-e on 
Human Resotuces; 

I appreciate the opporTiiuity to testify on the evahtarion of eftbits to help families support their children 
and escape poverty. As brief backgromid. the Coalition for Evidence-Based Policy is a nonprofit, 
nonpartisan organization, established in 2001 . We woifc with federal officials to increase the effectiveness 
of goveniment social spending through ngorous evidence about ‘"what works,” and the core ideas we have 
advanced have helped shape evidence-based reforms enacted into law and policy during both the Bush 
and Obama Administrations. We are not affiliated with any programs or program models, and have no 
financial interest in any of tlie policy ideas we support, so we serve as a neutral, independent resource to 
policy officials on evidence-based programs. Our work is funded primaiily by national philanthropic 
foundations, mcluding the MacAithiu' Foundation and the William T. Grant Foundation. 

My testimony will first discuss how evidence-based program refonns can greatly luciease the 
effectiveness of government social spending in improving people’s lives. Then 1 will offer some 
suggested next steps that the Subcommittee might consider to advance such reforms in the progi’ams 
witliin its jraisdiction. 

i. Rigorous randomized studies have the ability to identify program reforms (“interventions”) 
that increase the effectiveness of social spending while actually reducing its cost. 

It IS often assumed that the only way to increase government’s impact on social problems such as 
poverty and educational failure is to spend more money - an assumption that conflicts witii the 
current national interest in reducing the deficit. Largely overlooked, however, aie clear examples 
from welfare and other areas where rigorous randomized trials - widely considered the shxuigest 
method for evaluating program impact - have identified program reforms that produced important 
improvements in people's lives, while simultaneously reducing government spending. These 
examples suggest tliat a systematic government effort to build a body of such proven reforms, and 
disseminate them widely m federal social pmarams, could improve life outcomes for millions of 
Americans without adding to - or while even reducing - the deficit. 

Examples from programs within the subcommittee’s jurisdiction; 

■ Welfare-to-work strategies shown to produce sizable increases in participant’ employment 
and earnings, and reductions in their use of government assistance, hi the 19<S0s and 1990s. 

govemmeiit. foundation.s. and leading researchers sponsored or earned out a large number of 
randomized conti'olled trials of state and local welfare reforms. Three major refomi efforts - rtvo 
m Califoima, one m Oregon — were found especially effective. Focused on movmg welfare 
recipients quickly into the workforce rlirough short- tenii job-search assistance and taining (as 
oppo.sed to longer-term lemedial education), The mitiatives produced gains in participants’ 
emplojment and eaniings of 20-.50?'o. Remarkab] 5 ^ Urey also produced net savings to the 
goveminent, m reduced welfaie and food stamps, of SI, 700 to S6,000 per person.^ 
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These findings helped bmld political consensus for the strong work requirements m the 1996 
welfare reform act, and shape many of the work-fii'st state-level refonns that followed. The 
scientific rigor of die findings were critical to dieh policy impact." 

Subsidized Guardianship - an innovation in the foster care system shown to increase 
children’s piacement in a permanent home white reducing foster care spending. In die late 
1990s, the Depaitment of Health and Human Services fHHS) granted Illiuois a waiver from 
federal law to implement Subsidized Guaidirinship — an alteiuative to foster care ui which the state 
pays a siibsK^’ to a child’s relative or foster parent to become the child’s legal guardian. The 
approach is similai' to adoption but, milike adoption, does not reqime the tennination of parental 
rights and so can be easier to achieve from a legal standpoint. 

As a condition of the waiver, Illmois evaluated Subsidized Giiardiaiibliip in a large landoimzed tnal.' 
Over a nine-year peiiod, die study found dial the program (i) increased children’s placement in a 
permanent home by 8%; (ii) reduced average days iii foster care by 16%; and (in) produced net 
govenimeiit savings of S2,300 per child (because subsidizmg a guaidian is administratively less costly 
than foster care). Based on these findings and siiccessfiil replication tiials in other states, CBO scored 
sa\niigs of $800 million for federal legislation, enacted in 2008, to expand Subsidized Cniardianship 
nationally.'* 

Reemployment and Eiigibiiity Assessments - an innovation in the Unemployment 
insurance (Ut) system shown to produce Ui savings while increasing workforce earnings. In 

2009 , the Department of Labor launched a four-state randomized trial of the Reemployment and 
Eligibilitv’ Assessment (REA) program for IT claimants.' Hie program includes a mandatoiy in- 
pei'son review of tiie claimant's eligibility’ for UI, and personalized job-seaich and other 
reemployment assistance. Over a 12-18 mouth period, the study found: (i) $180 in net goveromeuf 
savings per claimant from reduced UI payments; (n) especially large savings m Nevada — $604 per 
claimant - possibly due to distinctive featmes of Nevada’s REA program that could be replicated 
elsewhere; and (ni) an uiciease in job eaiiimgs of $2,600 (18%) per clauiiaiit m Nevada — tiie one 
site that obtained a reliable estimate of the effect on earnings. (Tlie study also found a smaller - 
5®/o — increase in eaniiugs in Florida over a 12-monlIi period, but the study's analysis suggests this 
finding may' not be reliable.^’) 

These results suggest that nationwide implementation of REA for all eligible UI claimants could 
produce $1.5 billion in net government savings per y’ear. while increasing workers’ earnings. If 
the larger' Nevada effects could be reproduced nationally, the savings might be as high as $5 
billion per year, ^ and the increase in workers’ earmngs could be substantial. 

Nurse-Family Partnership - a home visiting program for low-income, first-time mothers shown to 
produce major improvement in participants’ lives, while reducing their use of public assistance. 

The Nurse-Family' Partnership (NFP) is one of the mahi program models funded by HHS’s 
Maternal, Infrint, and Early Childhood Home Visiting program. NFP has been sho'svii in three well- 
conducted randomized tnals to produce major improvements in pai+icipaiits’ life outcomes, such 
as; (i) 20-50% reductions iti cliild abiise^heglect and injuries; (ii) 10-20'l-c> reductions in mothers' 
subsequent births dmuig then late teens and early twenties; and (m) sizable mqiiovements m 
cognitive and educational outcomes for children of tlie most at-risk motliers. hi adcUtioti to these 
benefits, newly-published reports fr'om the ongoing liial in Memphis, Tennessee show-', 12 years 
after the women gave birth, a $i,i 13 reduction in annual government spending per woman on 
welfare, food stamps, and Medicaid during tiie 12 years. As a result, the total discounted 
gor’eiTiment savmgs over tlie 12 years ($13,350) more than offset tlie progi'aiii’s cost ($12,493).*’ 
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ii. To identify enough of these reforms to generate broad-based gains in government 
effectiveness requires strategic triai-and-error - i.e., rigorously testing many promising 
interventions to identify the few that are effective. 

Rigoious evaluations, by measuriag progiams’ tiue effect on objectively important outcomes such as 
workforce earnings, college attendance, teen pie.gu8nc\\ and child maltie-atment, are able to distinguish 
those that produce sizable effects from those that do not. Such studies have identified a few 
inteiv'entious tliat are truly effective — such as those described above — but these are exceptions that 
have emerged from testing a much larger pool. Most, iiic hiding those thought promising based on initial 
studies, are found to produce few or no effects — imderscornig the need to test many. This is tine not 
only in social spending, but in other fields where rigorous evaluations have been carried out. For 
example; 

• Education: Of tiie 90 inteiventious evaluated in randomized tiials conmiissioned by the Institute 
of Education Sciences (lES) since 2002, approxunately 90‘?-'o were found to have weak or no 
posi tryeeffects . 

• Empioyment/training; Of the Id mfei\'ention.s evahiated m Depaitment of Labor randomized 
tiials that have repotted results since 1992. about 75*/p were fpuiid to liaye fpmid yyeak or no 
positive effects.” 

■ Medicine: Reviews iiave found that d0-80®4 of ppsiriye residts in initial (;'phase ir’) clinical 
studies are pyertumed in subsequent, more defimtive randomized tiials (“phase III”).'’ 

■ Business: Of 13,000 randomized trials of new products.-'strategies conducted by Google and 
Microsoft. 80-90% have reportei^y fomid no significant effects. ” 

<((. The current pace of rigorous testing is far too slow to build a meaningful number of 
proven-effective interventions to address our major social problems. Of the vast array of 

ongoing and iiewly-imtiated progiain activities ui federal, state, and local social spendmg, oniy a 
small finctiou are ever evaluated in a credible way to see if they work. For example, based on our 
carefiii inonitormg of the hlerature, the federal government commissions randomized evaluations of 
only i-2 dozen such program activities each year. 

iV. The end goal - a sizable body of proven social Interventions - is of critical importance. In 
recent decades, the U.S. has failed to make significant progress in key areas such as - 

■ Poverty: The U.S. poveity rate — now at i5®'o — readied its low m 1973. It ha.s shown little change 
(whether by official or alternative National Academy measuies) .since tlie 1970s.” 

■ K-1 2 education; Readmg and math achievement of 1 7-year-olds - the end product of our K-12 
education system - is virtually imchanged over the past ^10 years, according to official measures,” 
despite a 90% increase in public spending per student (adjusted for mflatioii).” 

■ Welldjeing of low to moderate income Americans: The average yearly income of die bottom 
40% of U.S. households, now at 520,221, has clianged httie since the early 1970s.' 
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V. We therefore urge the Subcommittee to lead a bipartisan reinvention of U.S. social 
spending based on evidence about “what works,” through steps such as the following; 

A. Authorize and encourage the agencies to make maumum use of waivers from federal iaw 
and regulation to build credible evidence. 

1. “Waiver-evaluations” were deployed with great success in 1980s/90s welfare refOTm, 
making a critical contribution to the body of welfare-to-work evidence discussed above. 

Specifically, in the yeais leading up to the 1996 welfare lefomi act - through botli Republican and 
Democratic Administi'arion& - OMB and HHS had in place a “denion&tiation waiver" policy, under 
which HHS waived provisions of federal law and regulation to allow states to test new welfare 
reform strategies, but only if the states agreed to evaluate their lefomts in rigorous (usually 
randomized) studies. 

Tliis policy directly resulted in more th^ 20 large-scale randomized controlled hials that tested 
au important and diverse set of retbnus, and thereby helped build the influential body of welfare- 
to-work evidence discussed above. Hiese reforms that were tested include, for example, 
maiidatoiy' job search and eLuploymeirt actmties (e.g., Vermont); emplovTiieur subsidies for 
welfai'e recipients who left welfare for full-time work (e.g., Nevv York, Minnesota); time limits on 
welfare (e.g., Florida, Connecticut); “family cap" policies designed to discotnage additioniil biitlis 
among women on w'elfare (e.g., Aikansas, New Jersey'); and v'aaious combinations of the above 
reforms. 

2. We encourage the Subcommittee to advance a similar waiver-evaluation concept 
approach across the broad range of programs within its Jurisdiction, designed to: 

a. Stimulate a robust array of state/locai program innovations, aimed at (i) producing 
budget savings while improving program effe^iveness, or (ii) improving participar^t 
outcomes without added cost; and 

b. Require rigorous - preferably randomized - evaluations to determine which of these 
innovations really work. 

For some programs. This would require legislation to expand the program's waiver authority 
aud'or fie that authority to a reqiiiremeuf for rigorous evaluations wherever feasible. Other 
programs aheady liave sufficient authority, and die Subccirunittee could encourage them to use it 
more widely and stiategically to stimulate state/loca! innovation and evidence-building. We 
would be pleased to work with tiie Subcommittee, if helpful, to explore how the waiver- 
evaluation concept might he operationalized across various programs witiiin its jurisdiction. 

B. Authorize and encourage agencies to allow greater researcher access to administrative 
data, with appropriate privacy protections, so as to facilitate low-cost rigorous evaluations. 

1. Researchers have shown it is often possible to conduct randomized trials at low cost, by 
measuring study outcomes with administrative data already collected for other purposes. 

In a dev'elopiuent that could revolutionize .social policy' and practice, researchers have shown 
it is often possible to conduct .scientifically-rigorous evaluations of program effectiveness ot 
low cost, addaessmg a major obstacle fo their widespiead use. Costs are reduced by 
measuring study outcomes using administrative data ah eady collected for other purposes, 
such as child mallreanneiit rates, employinent and earnings, student lest scores, criminal 
arrests, receipt of government assistance, and health cai'e expenditur es. This elunmates what 
is typically the largest cost component of a rigorous study: locating each individual in the 
piogiam and control group at vuiious points in time after program completion, and 
adminisTeruig interviews or tests to obtain tlieir outcome data. 
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2. It is now possible to rigorously test hundreds of social program models and strategies 
each year, rather than a select few (as is currently done). As noted above, tl}e 

goveiunient already fimds a vast airav' of piogiam models' strategies each year, but only a 
small fi'action are ligoroiisly evaluated. With impetus from, policy officials, low-cost, rigorous 
ev'aluations could likely be embedded across a broad spectmm of such activities (recognizing 
that they may not be feasible in ev^’erj' area^'’). 

3. As iiiustrative examples - 

• The Subsidized Guardianship randomized trial, described above, cost a total of 
about $100,000 over nine years - because outcomes for the 2,400 children in the 
sample were measured with administrativ'e data on foster care outcomes (such as 
placement m a pennanent home) that the state of Ilimois already collected for other 
purposes. 

■ The Reemployment and Eiigibility Assessment trial, described above, cost about 

$320,000 through the 12-18 month foKow-up, based on ttte researchers' rough estimate. 

Even though tire study had a very large sample (135,000 UI claimants), it was conducted at 
modest cost by measmfng all otitcomes using administrative data on UI receipt and earnings 
tliat the participating states collect already for otlier purposes. 

> We recently developed a brief with five additional examples of trials costing between 
$50,000 and $300,000 - a fraction of the usual multimitlion-dottar cost for such studies. 

These studies all produced v'aiid evidence that is of policy and practical importance - and. in 
some cases, identified actionable strategies that generate budget savings.’^ 

4. We recognize that the Subcommittee has played a leadership role on this issue with its 
recent bipartisan approval of H.R. 1896 - increasing researcher access, with 
appropriate privacy protections, to HHS's National Directory of New Hires (NDNH). We 

believ'e tills legislation, if enacterl, will greatly low'er the cost and bui'den of conducting 
rigorous evaluations of employment pro,giaiiis, by enabling such studies to measure 
empioymient and earnings outcomes tluough NDNH data rather than engaging in costly new'" 
data collection (e.g., individual interview's). 

5. OMB is also seeking to advance greater use of iow-cost rigorous evaluation methods, 
and could be a valuable partner to the Subcommittee in advancing such studies. For 

example, OMB prominently' featured the concept of low-cost RC^s m its May 2012 memo to 
the heads of the federal agencies on ,f;se_of Evidence and Eyaluat^^ and cited the brief w'e 
developed on such studies. The President's FY 2014 budget does the same (link, page 94). 

6. Given the great potential, we encourage the Subcommittee to explore, through oversight 
hearings or other means, what more can be done by Congress andfor the agencies to 
build, integrate, and facilitate researcher access to administrative data, so a& to enable low- 
cosi, rigoi'ous evahiatioiLS acro.ss abroad range of proaiani areas. 

C. Once an intervention has been proven through rigorous evaluations to improve participant 
outcomes and/or reduce cost, authorize agency programs to use waivers and other 
administrative actions to facilitate its widespread adoption with program funds (wliiie 
ensuiTiig close adlierence to the proven approach). 
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Tlie reason we suggest this is that federal social programs generally do not have the statutory' 
authority to use evidence of effectiveness as a key criterion for allocating program hinds. (An 
iniporlaiif. but still relatively small, exception is the set of ’‘tiered evidence'’ initiative that 
Congress has enacted in recent yeais.’^) As a result, lesearch-proven, cost-saving interventions 
such as Reemployment and Eligibility Assessments and Subsidized Guaidianship, described 
above, may never be widely implemented luiless Coiigiess steps m to change tlie authonzmg 
legislation for the III program (in the case of Reemploynient and Eligibility Assessments"') or the 
Title IV-E Foster Care program (in the case of Subsidized Guardianship). C'ongrsss did act m the 
case of Subsidized Guardian.ship. as noted earlier, but this can be an imu.suat occurrence and, 
when it does happen, it often takes years to achieve. 

If Congress were to authorize programs such as UI and Foster Care to take admitiistrative action 
To foster wide implementation of proven, cost-saving inreiventions, it would create a much more 
efficient meclianism for tianslatmg credible research findings into practice, so as to improve the 
lives of program paitic^Jants and/or produce savings to tlie taxpayer. Doing so would inject a 
dyiiaimc for evideiice-diiven improvements into a social spending process where evidence 
cmrently has little role. 

Conclusion: Evidence-based policy offers a demonstrated path to more effective, less expensive 
government. As discussed in my testimony, we believe it could provide the basis for a bipartisan 
effort to reinvent U.S. social spending, so as to greatly increase its effectiveness in improving 
people’s lives. 
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Chairman REICHERT. Ms. Cox, please. 

STATEMENT OF KRISTEN COX, EXECUTIVE DIRECTOR, UTAH 
GOVERNOR’S OFFICE OF MANAGEMENT AND BUDGET 

Ms. COX. Thanks for having me here, Chairman and Ranking 
Member Doggett. It is an honor to be here. I am the executive di- 
rector of the Governor’s Office of Management and Budget, and 
prior to this position, I was the executive director of the Depart- 
ment of Work force Services. We oversaw the implementation and 
administration of over 90 different Federal programs, which in- 
cluded everything from TANF and food stamps to child care to 
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housing, a plethora of services that impact low-income individuals. 
I come to this discussion with a real on-the-ground perspective in 
how you actually operationalize evidence-based practices and pene- 
trate into the day-to-day work of our folks. 

Let me go through some of what we do and some things for you 
to think about. First of all, our goal in Utah is to improve the oper- 
ations of all of our systems in Utah State government by 25 per- 
cent over the next 3-1/2 years, the remainder of Governor Herbert’s 
administration. It is a bold initiative, but we think there is ample 
capacity in all of our systems to do better, and in social services, 
it really resides on integrating evidence-based practices into the 
day-to-day work of our employees, so they are spending more and 
more of their time doing what works, and less time doing the 
things that don’t or are wrapped up in compliance initiatives, 
which is part of the Federal bureaucracy. 

A few things, just observations we have before we get into the 
evidence-based practices. One, there is significant goal 
disalignment against — across services that serve low-income indi- 
viduals, so the ability to assess if a program has been successful 
or not is so dependent on the point and policy objective of the pro- 
gram that they are all over the place. For example, the Food Stamp 
Act 1964’s intended policy objectives was twofold, to promote the 
agricultural economy, and to give nutritional sustenance to low-in- 
come individuals. With amendments, the ABAP program, employ- 
ment and training services were offered, but really more as an eli- 
gibility criteria than a true strategy to move people to work. Add 
housing initiatives, TANF, Medicaid, across the board, the policy 
objectives are different, so when we really want to talk about the 
impact to low-income individuals, we need to be clear on what our 
intended purpose is, and we don’t have that right now. 

Second piece of concern is the ability of measurement. We have 
too many contradictory and conflicting measures out there in the 
public service arena. As an administrator of 90 different programs, 
trying to get clarity on how well I am doing is a challenge. We have 
created a very simple ratio, quality throughput divided by oper- 
ating expense, which will baseline and drive all of our performance 
in State government in Utah. It is not so simple in the Federal 
navigation system of measurements, even in common core. In Utah, 
we were very aggressive about understanding how important evi- 
dence-based practices was. When I was executive director of Work 
force Services, we set up an evidence-based arm of our agency spe- 
cifically to do randomized sampling, propensity scoring, everything 
we needed to know to analyze and assess TANF participation, job 
training, does it work or not, but what we found is what the na- 
tional studies say on a very universal level aren’t necessarily true 
for the unique demographics in Utah. Even within Utah, we saw 
variations from region to region. 

So while evidence-based practices at the national scale are im- 
portant to help direct Federal policy, the States need the ability, 
flexibility, and resources to create that same ability at the local 
level to really fine-tune and penetrate those evidence-based prac- 
tices into our system. 

The next piece is penetrating evidence-based practices into our 
operations. It is great to have theory, it is great to have tons of 
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data, you can Google and you can find hundreds of social service 
evidence-based practice reports, but why aren’t they penetrating 
our system? Part of it is our folks are so heavily focused on compli- 
ance activities that they don’t have the time to step back and think 
about what should we be doing. It is not an excuse, but it is a re- 
ality for people. Operationalizing takes the ability to translate glob- 
al goals into the day-to-day work of our employees, it requires that 
we have clear policy objectives for them, it really requires that we 
stop doing the stuff that doesn’t work and start doing what does. 
My hope is for every new policy initiative Congress puts out, they 
eliminate another one. There is only 8 hours in a day, it is the em- 
ployee’s biggest constraint. If you want them to really spend 80 
percent, 90 percent of the time that makes the biggest difference, 
then we have got to strip away the 70 percent of the stuff that is 
junk. 

A few suggestions or hopes or recommendations. A lot of dem- 
onstration projects are going on at the national level. Fantastic. 
Push some of those resources to the State level so we can customize 
our own practices that we need to make it relevant for us. Align 
the policy objectives, which is simple. Not simple, but critical. Sim- 
plify the measures, as you said so well, give States more flexibility 
in the ability to innovate, hold us accountable, but give us the abil- 
ity to be innovative. Thank you very much. 

Chairman REICHERT. Thank you, Ms. Cox. 

[The prepared statement of Ms. Cox follows:] 
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July 17,2013 

PREPARED TESTIMONY FOR THE RECORD of the: 

U.S. HOUSE COMMITTEE ON WAYS AND MEANS, SUBCOMMITTEE ON HUMAN 
RESOURCES 

Offered by Kristen Cox, Executive Director 

Governor's Office of Maiiagemenl and Budget, State of Utah 

Thank you. Chairman Reichert, Ranking Member Doggett and Members of the Committee. I 
appreciate the opportimity to appear before you today to share my observations relative to the 
effectiveness of public assistance programs. I am Kristen Cox, Utah's Executive Director of the 
Governor’s Office of Management and Budget. 

In Utah, the Governor s Office of Management tmd Budget (COMB) oversees the initiatives of 
the Governor regarding budgeting and planning across all state agencies. GOMB has recently 
been tasked by Governor Gary R. HeiUert to improve state government operations and services 
by 25 percent by December 2016. 

The Governor was clear in his directive that the 25 percent target is not tui exercise in simply 
reducing budgets; ratlier the focus is on improving all aspects of operational perform<mce. 

In support of this goal, GOMB has developed a comprehensive set of operational excellence 
tools and principles — called the .SUCCESS Framework. The SUCCESS Framework is gi'ounded 
in seven fundamentals of high peifonning organizations: 

• Set raeasureable goals and targets 

• Use thinking tools 

• Create your strategy 

• Create your organization 

• Engage staff at all levels 

• Synchronize policy cind projects 

• Stay focused 


Additionally, GOMB has developed a perfornumce measurement system to track overall 
progress and recognize results — results that Ccin be used for making better management 
decisions. Ultimately, the SUCCESS strategy will keep Utah on lop as the best managed state in 
the nation, continually delivering value to customers and taxpayers. 

In government, we know- that evidence-based practices can be a solid foundation upon which to 
design and target resources. However, programs or services built upon evidence-based logic 
models do pose challeiiges. In many of these programs, the stated target is a longitudinal goal 
and it is not uncommon for many of them to lack sufficient control over the ultimate ends of their 
effort. They also tend to be measured at insufficient intermediate milestones, leaving proximate 
gaps of evaluation. These gaps make it difficult to manage the program, understand the core 
v^aiue of the investment or determine what adjustments (if any) are required. A few years ago. 
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when I served as the Executive Director of die Depailment of Workforce Services, we designed 
our Workforce Research and Analysis division to be die evidenced-based anii of our agency, 
supporting key decisions with real-time, measurable data. 

In Utah, we iire working on solutions to identify and close these proximate gaps in peiformtuice 
measurement. Some of our targets of consideration: 

• Understanding that not all systems or programs are the same, If an effort has been 
proven by evidence-based research on a large scale then a similar patteni of milestones 
could, but not always will, be replicated in a smaller pool for implementation. These 
milestones would need to be developed generally and modified specifically. 

• Developing measurement strategies that focus resources only on components of a service 
or program that actually acliieve the end goal. 

• Redefining the measurement of value delivery for some services dial is sensible to 
stakeholders and to the taxpayei'. 

• Measuring Quality Throughput (accurate, timely, effective service) over Operating 
Expense, or QT/OE 

Utah believes that measures should be simple and take into account qucility, throughput/volume, 
and costs. We can't look at any one of these elements in isolation from the others. 

Improvements to quality and volume arc relative to the cost of such improvements. Likewise, 
reducing costs may or may not be a good thing depending on the impact to quality and volume. 

In Utah, our performance management system is based on thi s very simple model that ensures 
we are looking at the elements in relationship to one another, 

Using our management system, each cabinet agency is on target to achieve Governor Herbert’s 
25 percent improvement goal. 

As w’e look across the federal programs that are managed by the State of Utah, it is clear that not 
all of them are planned or measured consistently. One of the largest agencies in Utah is the 
Depailment of Workforce Services, which manages over 90 federal programs designed to 
provide public assistance. If proper execution of these programs is truly to be defined by quality 
throughput (job attainment) then localized, evidence-based practices and strategies should drive 
federal policy. 

Utah believes that iill government programs should be stmetured in a miumer that maximizes 
resources for participants iuid drat requirements should be minimized and focused on developing 
a consistent and structured performance accountability system which measures evidence at 
designated milestones, 

During my term as Executive Director at Workforce Services, I commissioned studies aimed at 
better understanding the outcomes associated with the Temporary Assistance for Needy Families 
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(TANF) work participation requirements, job training programs and intergenerational poverty in 
Utah. We have invested resources to quantify and evaluate programs designed to help parents 
and childien to escape poverty. ’J'hese studies assist lawmakers, executives and program 
managers to identify and implement effective strategies to seiwe these families. Ut<ih 
understands that policy concentiation must be centered on the atuiiiiment of economic self- 
sufficiency with results focused on employment outcomes. Notable studies in this regard 
include: 

• TANF - Meeting Welfare’s Work Panicipation Requirements and Transitioning into the 
Labor Market, (Kj'anlz and Torosyan, DWS, September 2012): The longitudinal effects 
of activities associated w ith TANF w'ork participation are estimated using ordiiuuy least 
squares regression and include socioeconomic characteristics fdong with employment 
patterns. The paper explores services that correlate with successful pailicipation in the 
TANF program and how services link with post-TANF earnings. 'ITie study shows that 
individuals with participation in employment related services are most likely to meet 
participation requirements and that those who meet participation most often have tlie 
greatest attachment to the labor market after TANF. Tiie paper is available at: 
http://iobs.utah.gOv/wd/piibs/specialreports/tanfreport092012.pdf 

• Work Success - Family Employment Program- {FEP) Redesign Study of Utah 2012: 

Wave I (Vogei-Ferguson, Social Research Institute. College of Social Work, University 
of Utah, December 2012): This study examines outcomes of the Utiih Work Success 
program using customer interviews and administrative data. ’‘The Work Success Program 
was designed in response to customer requests for more help in finding and retaining 
employment. By far, most customers are very pleased with Work Success and have found 
it to be an effective program for filling this need. Most customers w'ho are referred to 
Work Success iire ready and available for W'ork and are able to engage in and benefit 
from the services. . . Most participants have adequate skills for the level of computer skills 
needed, most are able to secure childcare and transportation, and . . . nearly half iue 
employed when they leave the program.'* The paper is available at: 
http://www.socwk.utah.edu/sii/pdf/DW.S__2Q12__FEPRedesignRepoft.pdf 

• Job Training Programs - (s Job Training Justified: An Analysis of Job Training Services 
as Administered by Utah’s Department of Workforce Senices, (Krantz and Mayne. DWS, 
August 201 1 ): Tills experimental study uses propensity score matching to analyze the 
treatment effects of job training seivices in Utah. Examining these issues helps guide 
DWS policy-makers to more effective budget and public service decision-making. 
Findings help identify specifre job training services that with the most employment and 
the highest eaniings. The paper is available at: 

• Intergenerational Poverty - Intergenerational Poverty in Utah 2012, (Little, DWS, 
September 2012): This report uses administrative data to provide descriptive statistics of 
adults in Utah with public assistance (TANF, Food Stamps, and Medicaid) w^ho also 
received public assistance as children. The report describes the demographic and 
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socioeconomic characteristics of these individuals including education and employment 
history. The paper includes demographic information for the children of these adults. 

The paper is available at: http://iobs.utah.gov/wi/pubs/PovertY .Repoit .web.pdf 

In Utah, more than 70 percent of all |>eopJe living in poverty receive some form of public 
assistance including financial aid, child care subsidies, SNAP, and/or Medicaid. One-third of 
adults in poverty in Utah have less than a high school diploma or CED. Most of the remaining 
population has no post-secondary education and limited work history. Our studies do show that 
recipients of job training programs are exponentially more successful when they successfully 
complete the training: yet less than 50% of all recipients of training actually complete the 
training, Programs such as completion incentives and intensive counseling aird monitoring could 
prove to be w'orlhwhile expenditures, but states must have the flexibility to execute strategies 
that work, rather tlian spend significant resources on administrative bureaucracy. 

The current design of the public w'orkforce investment system is a maze of individual programs 
and funding streams with various mandates attached to each program. It is the expectation of the 
states to manage through these mandates and bureaucracy and provide the employment and job 
training services needed by the workforce. This has proven to be a climate that is not always 
conducive to a customer's success in achieving self-sustainability. For example, translating 
evideiiced-based practices into the day-to-day work of an employee t<ikes considerable effort and 
focus. It starts with trying to eliminate tlie things that prohibit or disteact the employee from 
spending time on the tilings that count the most. Witli only eight hours in a day, it is impoilant 
for federal policy makers to be mindftil of everything they are asking people to spend their scarce 
time and energy on. Our biggest constraint on the ground is time. Aligning our policies and 
laws around the reality of time and how we want people to use it is a much more realistic way to 
bring evidenced based practices into reality. 

In addition, policy objectives of various federal programs are different and can be challenging to 
know w'hich evidenced-based practice to apply when serving an individual who is receiving 
multiple services. Likewise, the customer receives mixed messages and finds navigating 
multiple programs difficult. For example, food and health care are both critical needs. However, 
tlie SNAP progi'am requires, in certain cases, that people engage in work activities while 
Medicaid does not, Churifying the policy objectives and niles across the spectrum of programs 
intended to serve a specific population would translate into a more focused use of resources tuid 
effort on tlic ground. 

When properly aligned, program integrity efforts, re-employment initiatives, operational 
efficiencies, and trust fund management should ensure that limited resources are maximized and 
directed to those who are eligible for assistance and re-employment activities. Utah has an 
integrated model that captures over 90 different iederal programs, giving us a unique and 
comprehensive perspective on employing individuals. In fact, a Government Accounting Office 
(GAO) report recently singled out Utali for our consolidation efforts and noted that “tire 
consolidation allowed job seekers to apply for assistance they had not considered in the past.’' 
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In Utah, we are coimnitted to assessing tlie quality of programs adiniiiistered and are proactively 
reviewing sendees in order to ensure maximum value is provided to the public. 

As this committee continues its important work, Utah lespectfully suggests that the following 
core principles guide reform efforts focused on public assistance programs that lead to 
employment: 

1 . Programs should be structured in a manner that maximizes resources for participants; 

2. Requirements should be minimized and focused on developing a consistent and 
structured performance accountability system; 

3. Stales should be provided maximum flexibility to design the programs and initiatives 
best suited to its citizens, businesses, and workforce development partners; 

4. Budget streamlining should not just penalize the states — federtil agencies should be 
examined and unnecessary bureaucracy and processes should be eliminated; 

5. Innovation and risk-taking in the design and delivery of employment and job training 
seivnces should be encouraged rather than penalized; 

6. Programming should be data-driven and evidence-based with tangible accountability 
measures; and 

7. Congress must refrain from establishing parallel job Tniiniiig programs and/or 
discretionary grants tliat duplicate the existing workforce system. 

A specific area where Congress can help promote efficiency, better serve job seekers, workers, 
youth and employers, and maintain a level of services with fewer financial resources is to 
provide states with a new Workforce Investment Fund which would be an integrated grant to 
states that combines the follow'ing current individual formulaic grants: 

• Workforce Investment Act Adult 

• Workforce Investment Act Dislocated Worker 

• Workforce investment Act Youtli 

• Wagner-Peyser Employment Seiwice 

These four I'unding streams provide the foundation for the Workforce Investment Fund because 
they provide the same or similar services which could be enhiuiced to populations needing 
employment and training assistiuice. 

In addition and at the request of the Governor, the following programs could be delivered 
tlirough a new- Innovation Waiver process through the W'orkforce Investment Fund: 

• Adult Education 

• Vocational Rehabilitation 

• Trade Adjustment Assistance (trainiag) 

• Veterans Employment and Training 

• Food Stamp Employment and Training 

• Temporaiy Assistance for Needy Families (employment/training) 
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The Innovation Waiver process would involve the appropriate Cabinet Secretary in charge of the 
program and would provide a state with an opportunity to demonstrate how^ delivery of the 
program would pi'omote efficiency and improved services for customers and set a common 
standard for participation. Waiver requests w'ould need to be responded to within 30 days or the 
waiver request would be automatically approved, In addition, the waiver process should also 
allow states to include strategies that would better integrate and align Unemployment Insurance 
{'T.H) customers into the broader workforce system. Traditionally, UT re-empioynient effoits are 
isolated from the broader system and are often nonexistent in many states. 

Utah also believes that Congress should decrease the number and amount of discretionary grants 
overseen by DOL and opt for funds with clear accountability standards. Slate and local 
governments spend too much time and resources on “chasing” money in the form of grants that 
nmy not best be suited for their unique needs. With diminishing resources, it’s unfortunate that 
state and locaJ governments arc increasingly faced with the dilemma of hiring full-time grant 
writers or bringing consultants on boctrd who are well-versed in how to navigate die grant 
process, Consequently, grant awards can be made on how w'ell the application is written rather 
than on the actual merits of the proposal. Grant writing has become its own cottage industiy. 

In addition, grants require separate budgeting, monitoring, and reporting — all of which take 
away money fmm customers and expand administrative overhead. Grants can lake too much 
time to approve and often end up being one-time programs with no prospects of sustainability. 
.States need resources they can count on to develop mciuiingtiil programs that can measurably 
move the needle over time and quickly respond to structural changes. 

Discretionary grant programs such as the Workforce Innovation Fund would be eJiminated in 
order to maximize funding to the states. Utah feels that directing any portion of federal funding 
currently set-aside as statewide activity funds for state-led innovations to a new federally 
dictated, controlled and prescribed program (such as the Innovation Fund) adds bureaucracy and 
defeats its intended purpose. 1 mainUiin that goveraors, not the federal govermnent, are uniquely 
positioned to innovate and advance systemic workforce development initiatives. Washington, 
D.C. should not be determining what is or is not innovative in Utah - the decision should be 
made by Utah’s Governor, 

As our nation struggles with reducing its debt while providing critical services, we must ask 
oursol ves how die taxpayer w'ould define “value” and if they w'ould be willing to pay for it. I 
suggest that many of the procedural aspects of federal policy could not pass diis test. Hoivever, 
at its core, public assistance and employment strategies offer significant value to the customer 
and to our nation as it elevates the competitiveness and economic prosperity of ou)' workforce. 

ITie State of Utah stands ready to assist the Committee in its efforts to bring innovative policy 
answers that aggressively address the re-employment strategies. We believe that states are the 
appropriate starting point for diese conversations and encourage you to maximize flexibility and 
allow states to focus on helping people find employment and then hold them accountable for 
doing so. Thank you for the opportunity to address the Committee and I look forward to 
answering any questions you may have. 
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Chairman REICHERT. Mr. Aos. 

STATEMENT OF STEVE AOS, DIRECTOR, WASHINGTON STATE 
INSTITUTE FOR PUBLIC POLICY 

Mr. AOS. Mr. Chairman, Members of the Committee, my name 
is Steve Aos, and I am the director of the Washington State Insti- 
tute for Public Policy. You asked me to come today and provide tes- 
timony on how the Washington State legislature, another legisla- 
tive body, is using the Institute to try to reform public policies in 
the State of Washington, so I am going to give a little bit of an 
overview of the Institute and how the legislature uses it and then 
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talk about some of the specific applications of the approach in 
Washington. 

The Institute was created 30 years ago in 1983. The legislature 
passed a joint resolution in that session and said we want to have 
an institute to help the legislative branch of government figure 
some things out. The key aspect of the Institute is that all of the 
assignments to us, including the ones I will talk about today, come 
to us because the legislature passes a study bill, goes through both 
Houses, passes both Houses, and is signed by the Governor. We 
don’t respond to an individual Member’s request, but a bill has to 
go through and pass for it to have a study undertaken. And in re- 
cent years what the legislature has been asking us to do, in about 
the last 15 years, is to find out what works and what doesn’t work 
to achieve particular public policy outcomes. So we will get a direc- 
tion that will say how can Washington public policy affect the 
crime rate in Washington State, or the rate of child abuse and ne- 
glect, or how can we get greater high school graduation levels in 
Washington State, and it will say to the Institute, tell us what 
works and what doesn’t to achieve those outcomes. 

It is a nonpartisan group, an equal number of Republicans and 
Democrats are on our board of directors, and they are cochaired by 
one Republican and one Democrat at all times, so it was set up to 
be a nonpartisan group, and that is how we operate. 

What works and what doesn’t work has been something that the 
legislature has been finding particularly attractive in learning how 
to do the numbers on the one hand and the legislative process has 
been learning how to use the numbers to actually craft budgets and 
policies in an increasing number of important areas. When we do 
this work, when we get that assignment to say what works, we are 
playing the role of the investment adviser, if you will. We make 
buy and sell decisions to our legislature, and we look all around the 
country and all around the world at all of the most rigorous evalua- 
tions on a given topic. We throw out evaluations that we think 
aren’t rigorous enough to warrant any further consideration, and 
then we assess all that in a systematic way, and we do a cost-ben- 
efit analysis, and we come back to our legislature saying this thing 
looks like it is a winner, this thing is maybe iffy, and this thing 
looks like a loser in terms of benefit cost. 

You mentioned the Scared Straight program in your opening re- 
marks. We have got lots of losers. We love to find losers because 
then if we are already funding those kinds of programs in Wash- 
ington, they have become things that we can then cut in terms of 
programs that are ineffective. So that is the role that the legisla- 
ture has had us do consistently over time. The hallmark of our 
work has been to take not only what works but actually to do a 
benefit-cost analysis of each of those of that effort. 

Crime policy has been the area where we have moved it the fur- 
thest, but we are moving ahead in K-12 education and child wel- 
fare and some of the topics that are directly before this Committee 
right now. We now actually can point to lower crime rates in the 
State of Washington and the reduced level of taxpayer spending in 
the State of Washington as a result of all that work and all the 
previous budgetary decisions that have been made from that work. 
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The latest approach in Washington, passed unanimously in the 
last two sessions, has been to take the Institute’s list of what works 
and what doesn’t work and send a message to the executive agen- 
cies saying align everything that you do and tell us what approxi- 
mates the Institute’s list and what isn’t on the Institute’s list, and 
those reports will then come back to the legislature from the execu- 
tive agencies in about five or six different areas of public policy. So 
this is the legislature’s attempt to try to take the information we 
have been doing and actually craft budgets around it by giving the 
executive branch a time to respond, saying are you doing things 
that the Institute has found to work or not work. 

Mr. Chairman, I just wanted to give an overview today about 
how the legislature back in your home State has been using this 
information. By the way, it is 68 degrees back in your home State 
today, I just checked. I am happy to get back there in a few hours. 
It is real progress. Session by session, we get better and better at 
doing the work, the legislature gets used to asking the question 
and taking the information back, and it is a nonpartisan effort, 
that is the thing that is perhaps most encouraging. Thank you for 
allowing me to testify. 

Chairman REICHERT. Thank you. I think 68 is the humidity 
level here in DC. if I am not mistaken. At least. 

Thank you for your testimony, Mr. Aos. 

[The prepared statement of Mr. Aos follows:] 
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Chairman Reichert and Members of the Committee, my name is Steve Aos and ! am the Director 
of the Washington State institute for Public Policy. You asked me to provide testimony today 
describing the nature of the institute and how it is being used In Washington state. 

Mr. Chairman, state policymakers in our home state have been engaged in a quite focused effort to 
make government better in a number of significant policy areas, in broad brush, the effort is to 
deliver to taxpayers better public policy outcomes and a more efficient use of their tax dollars, in 
business-like terms, policymakers are trying to provide taxpayers in Washington state with an 
improved return on Investment in certain public programs and policies. 

Structure of the Institute. Before i discuss the substance of the approach, you asked 8iat ! say a 
few words about the structure of the institute. The Washington State Institute for Public Policy was 
created by the 1383 Legislature. The purpose is to carry out non-partisan research on studies 
assigned to the Institute by the legislature. All of the assignments that we undertake come to us 
because the legisiature passes a bill, signed by the Governor, directing a particular study with 
specific study language along with an appropriation to carry out the study. This is a key feature of 
the Institute, institute staff does not cany out studies at the request of individual Members of the 
legisiature, nor does the staff undertake studies on their own volition. If the legislature passes a 
study bill and the Governor signs it, then the institute carries out the research. 

The institute is governed by a sixteen-member Board of Directors. Ten of the members are from 
the legislative branch; four Senators and four House Members and there are an equal number of 
Republicans and Democrats from each body. In addition, the two non-partisan committee staff 
directors of the Senate and House round out the 10 Board members from the legislative branch. 
The Governor appoints h/vo cabinet level members to Institute's Board, and the remaining four 
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Institute Board Members are from the public universities and colleges in Washington state. The 
Board is co-chaired by a two rotating Senate and House Members, one a Democrat and one a 
Republican. 

The Institute’s Board hires the Director. Additionally, when an outside non-iegislative entity 
requests the Institute to undertake a study, the Board takes a vote. For example, sometimes a 
Washington State executive agency asks the Institute’s Board to undertake a study. Occasionally 
an outside entity such as the MacArthur Foundation, the Pew Charitable Trusts, or the Gates 
Foundation makes a request to the Board to fund a particular study. The Board meets three times 
a year to vote on these requests. 

Almost all of what we do, however, originates in study requests from the legislature itself. 

Two Types of Studies. The legislature assigns two types of studies to the Institute: (a) reviews of 
research from around the country where the goal is to identify potentially effective policies that 
could be put in place in Washington state, and (b) specific outcome evaluations of programs 
operating in Washington. By way of analogy, for the first type of study, we play the role of an 
“investment advisor” where we make “buy or sell” recommendations to the legislature based upon 
our examination of the options that have been tried and rigorously tested somewhere in the United 
States or beyond. Then, for the second type of study, we evaluate specific outcomes of 
Washington programs that policymakers in our state have put in place. 

We Focus on High-Quaiity Research and Benefits and Costs (Return on Investment). In the 

last fifteen years, the legislature has increasingly directed studies to the Institute of the first type. 
That is, we receive legislative assignments asking Institute staff to identify “what works?” and ‘what 
does not wo!l<?” to achieve particular public policy outcomes. We have, for example, carried out 
these “what works?" assignments in the areas of crime policy, juvenile justice, K-12 education, 
early education and preventative intervention, child welfare, and mental health and substance 
abuse. We are now conducting work on health care. We have examined a few of the early 
childhood options that are in the area of today’s hearing. 
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When we carry out these “investment advisor” studies for the legislature, we follow three specific 
research steps. 

First, we identify “what works” and ”\Atiat does not work” by analyzing all of the most credible 
research studies that have been done on a particular policy-relevant topic. For example, when the 
Washington legislative has asked us to identify “what works” to reduce crime, we examine ail of 
most credible studies that have evaluated sentencing, policing, adult corrections, juvenile justice, 
and preventative policy and program options. 

I emphasize the words “ail” and '‘most credible.” When we conduct these reviews we do not 
cherry-pick studies. If one wants to find a study or two showing that something works or does not 
work, one can usually find at least some studies pointing to either result. We avoid this selective 
reading of the literature by including all of the studies on a topic. We tfien deliver to our legislature 
conclusions based on file weight of the evidence from ail of the studies. The only requirement that 
we place on these reviews is that the studies included in our reviews must be based on credible 
scientific evidence. Sadly, a lot of research is done that does not pass our test of scientific rigor. 
Fortunately, however, for quite a number of public policy topics, there are enough credible studies 
from which reasonable conclusions can be drav/n. As I noted, our goal is to provide the legislature 
with ‘buy and sel!" informafion, so a finding that something doesn’t work can be just as useful to 
policymaking as a finding that something works, in addition, when we discover that there is not 
enough credible research on a particular topic upon which we can draw a conclusion, we also 
provide that finding to the legislature. 

Second, after we review the literature on what works and what does not, we then apply a benefit- 
cost test to the result to identify whether a policy option makes economic sense. Significantly, 
because we use a consistent way to measure benefits and costs across all policy options, all of our 
results can be ranked, based on comparative return-on-investment information. We then assemble 
the information In a “Consumer Reports” like listing of policy options. The reports have a 
consistent tabular structure — the same approach that Consumer Reports uses to evaluate 
products. This enables legislators on different policy committees to draw on a familiar looking 
analysis to help them make policy and budgetary decisions. 
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Our third research step involves assembling “portfolios” of options that legislators can use to 
address a particular outcome. Again, to use the crime example, if the legislature wants to reduce 
crime and do so in a cost-efficient manner a number of policy options identified on our “Consumer 
Reporte” like lists can be combined into a policy portfolio. Beginning in the late 1990s. the 
Washington legislature began using this information to assemble a crime-fighting portfolio and 
write budgets that involve a combination of sentencing, adult corrections, juvenile justice, and 
prevention policy options. 

An underlying analytical theme of our work addresses the risk involved in our estimates. Virtually 
all of the single-point, bottom-line results on our lists are the product of a number of estimates and 
assumptions. The world is risky. We employ statistical techniques that allow us to assess the 
likely range of outcomes in the information we forward to the legislature. That is, we provide the 
legislature with two basic findings for each policy topic: (a) our best estimate of a specific benefit- 
cost result, and (b) the chance that a program or policy will not at least break even. This is the 
same sort of information that venture capitalists want when evaluating an investment; an expected 
return on investment, and the probability that the investment will fail. We attempt to provide the 
Washington legislature with this same sort of information. 

Examples. I believe your staff has given you a copy of one of our reports to the legislature, 

“Return on Investment: Evidence-Based Options to Improve Statewide Outcomes, April 2012 
Update.” That report, along with ail other institute reports, can be downloaded from our website. 

http;//'ww\v.wsipp.vv3.gov/pub.3sp?docid=12-04-1201 

The legislature has been using the Information for a number of years, most notably in the area of 
crime poiicy. Those crime-related assignments were the first ones directed to the Institute by the 
legislature, so we are farthest along, analytically, in that important public policy area. In particular, 
over the last fifteen years, tie legislature has Increasingly used the Infonriation to both stop funding 
some programs for which the evidence indicates a poor return on Investment, and to implement a 
portfolio of evidence^Dased crime fighting strategies. One can examine the actual budgets that 
have been enacted in Washington in recent years and see the direct budgetary references to the 
Institute’s lists for the specific funding levels. We estimate that crime Is lower in Washington 
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because of It, and that there are about 1 ,200 fewer people in prison today in Washington (than 
there otheiwise would have been) because of the iower crime. 

Legislation enacted in more recent years from the Washington legislature is extending these sorts 
of initiatives to ttte areas of K-12 education and eariy education, children’s mental health and child 
welfare, adult mental health and substance abuse. Members of the Washington legislature have 
expressed interest in early intervention as a way to promote positive outcomes such as increased 
human capital and economic productivity, and as a way to avoid undesirable outcomes such as 
child abuse, crime, and rapidly increasing health care costs. 

In addition to these comments about this work going on in Washington State by the institute, the 
Pew Charitable Trust and the MacArthur Foundation has an initiative named “Results Firet.” That 
effort is designed to help other states assess the costs and benefits of policy options and use the 
information to help make decisions. The Results First project is using some of the analytical 
models created by the Washington State institute for Public Policy. The Results First states 
currently include: Connecticut. Florida, Idaho, Illinois, Iowa, Kansas. Massachusetts, Mississippi. 
New Mexico, New York, Oregon. Rhode Island, Texas, and Vermont. 

Conclusion. Mr. Chairman, you asked me today to speak generally about a process the 
Washington legislature is using to try to obtain more “bang for the buck’’ in policy making. Let me 
close by simply reiterating what I think we’ve learned in Washington state with this home-grown 
effort to try to improve public policy outcomes. We have learned that in some public policy areas, 
there is sufficient credible evidence from which reasonable “going-forward” conclusions can be 
drawn. We have found that this evidence indicates that some policies work to achieve outcomes; 
some policies do not work; and for some policy options, too little is presently known. We have also 
learned that it is possible to take a consistent business-like approach, using benefit-cost and risk 
analysis, to offer policymakers — state legislators in my case — with information to identify policies 
that are more likely to be successful. 

Washington state embarked on this approach about fifteen years ago and it is now applying it to a 
growing number of policy areas, including some of those that are the subject of your hearing today. 
My comments today have been directed to the process that we are using to supply the information 
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to policymakers in Washington. We will be happy to share with your staff more detail on the 
specific findings to date. 


Thank you for this opportunity to appear before your Committee. 
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Chairman REICHERT. Thank you, Mr. Muhlhausen, for being 
here, and you are up for 5 minutes. 

STATEMENT OF DAVID B. MUHLHAUSEN, PH.D., RESEARCH 
FELLOW, EMPIRICAL POLICY ANALYSIS, THE HERITAGE 
FOUNDATION 

Mr. MUHLHAUSEN. Thank you. My name is David 
Muhlhausen, I am a research fellow in empirical policy analysis in 
the Center for Data Analysis at The Heritage Eoundation. I thank 
Chairman Reichert, Ranking Member Doggett, and the rest of the 
Committee for the opportunity to testify today on the need to 
evaluate Eederal social programs. The views I express in this testi- 
mony are my own and should not be construed to represent any of- 
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ficial position of The Heritage Foundation. My testimony is based 
on my recently published book, Do Federal Social Programs Work? 

My spoken testimony will focus on three points: First, the best 
method for assessing the effectiveness of Federal social programs 
is large-scale, multi-site experimental impact evaluations that use 
random assignment. Unfortunately, these scientific rigorous assess- 
ments are rarely done. From my count, only 20 large-scale multi- 
site experimental impact evaluations assessing the effectiveness of 
21 Federal programs have been published since 1990. 

The consequence of so few Federal social programs being rigor- 
ously assessed means that Congress has no credible information on 
the performance of the majority of social programs. To solve this 
problem. Congress should specifically mandate the multi-site exper- 
imental evaluation of these programs. When Congress creates so- 
cial programs, the funding activities are intended to be spread out 
across the Nation. For this reason. Federal social programs should 
be assessed for national effectiveness. While an individual program 
operating at a single site may undergo an experimental evaluation, 
the small scale single site evaluation would not inform Federal pol- 
icymakers of the general effectiveness of the broader national pro- 
gram. 

The success of a single program that serves a particular jurisdic- 
tion or a population does not necessarily mean that the same pro- 
gram will achieve similar success in other jurisdictions or among 
different populations, thus small-scale evaluations are poor sub- 
stitutes for large-scale multi-site evaluations. A multi-site experi- 
mental evaluation that uses the performance of a program oper- 
ating in numerous and diverse settings will produce results that 
are more informative to policymakers. 

Second, the Federal Government does not have a good record of 
replicating successful social programs on a national scale. Policy- 
makers and advocates often assume that a social program that is 
effective in one setting will automatically produce the same result 
in other settings. This is a faulty assumption. For example, for the 
Center for Employment Training replication, the Federal Govern- 
ment attempted to replicate the successful outcomes of a youth job 
training program in San Jose, California, in 12 locations through- 
out the United States. A multi-site experimental evaluation found 
that the Federal Government was unable to replicate the successful 
outcomes in these other settings. Just because an innovative pro- 
gram appears to have worked in one location does not mean that 
the program can be effectively implemented on a larger scale. 

Third, policymakers should be mindful that Federal social pro- 
grams do occasionally produce harmful impacts on recipients. How- 
ever, social program advocates too frequently ignore these findings. 
Nevertheless, Congress should be aware of these harmful impacts. 
Here are two examples. From the 3-year-old cohort of the Head 
Start Impact Study, kindergarten teachers report that the math 
abilities of children ^ven access to Head Start were worse than 
similar children not given access to Head Start. 

Students participating in educational, after-school educational 
activities under the 21st century Learning Centers Program, were 
more likely to have disciplinary and behavioral problems such as 
getting suspended from school. Further, they were less likely to 
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achieve at high levels in class and were less likely to put effort into 
English classes. With the Federal debt reaching staggering heights, 
the hest method for assessing the effectiveness of social programs 
and making sure that money is spent wisely are large-scale multi- 
site experimental evaluations, yet to date, this method has been 
used in only a handful of Federal programs. Congress needs to re- 
verse this trend of not rigorously evaluating Federal social pro- 
grams. Thank you. 

Chairman REICHERT. Thank you, Mr. Muhlhausen. 

[The prepared statement of Mr. Muhlhausen follows:] 

Evaluating Federal Social Programs: Finding Out What Works and What Does Not 
T estimouy before 

the Committee on Ways and Means 
Subcommittee on Human Resources 
United States House of Representatives 

July 17, 2013 

Da^id B. Muhlhausen, Fh.D. 

Research Fellow in Empirical Policy Analysis 
The Heritage Foundation 

My name is David Muhlhausen. I am a Research Fellow in Empirical Policy Analysis in 
the Center for Data Analysis at The Heritage Foundation. I thank Chairman Dave Reichert, 

Ranking Member l.layd Doggett. and the rest of the committee for the opportunity to testify 
today on the need to evaluate fcdeml social programs. The views I express in this testimony are 
my own and should not be construed as representing any official position of The Heritage 
Foundation. 

My testimony is based on my recently published book, Do Federal Social Programs 
Work?^ This is a simple question. While the question may be straightforward, finding an answer 
is complicated. As my book demonstrates, the best method for assessing the etYectiveness of 
fedeiul social programs is large-scale, multisite experimental impact evaluations. Unfortunately, 
these scientifically rigorous assessments are rarely done. By my count, only 20 large-scale, 
muitisitc experimental impact evaluations assessing the effectiveness of 21 federal social 
programs hai’e been published since 1990: 

• Eai'ly Head Start" 

• Enhanced Early Head Start witli Employment Services^ 

• Head Start'* 

• Even Start Family Literacy Program' 

• 2 1 st C enlury’ Community Learning C enters'’ 

• Abstinence Education 

• Upward Bound^ 

• Food Stamp (renmned SNAP) Employment and Training Program^ 

• W'elfare-to-Work*^ 

• Employment Retention and Advancement (ERA) Project** 

• Building Strong Families*^ 

• Supporting Healthy Marriage*^ 

• Mo\dng to Opportunity and Section 8 Housing Vouchers*'* 

• Job Training Partnership Act (JTPA) programs*'^ 

• Unemployment Insurance (UI) Self-Employment Demonstrations*^ 

• Project GATE (Growing America Through Entrepreneurship)* ' 

• Job Corps*® 

• JOBSTART*^ 
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• Center for Employment Training (C'ET) Rq?lication^^ 

• Quantum Opportunity Program Demonstration"^ 

The results of these evaluations are summarized in Do Federal Social Programs Work?~^ 

The consequence of so few federal social programs being rigorously assessed for 
effectiveness means that Congress has no credible inlomiation on the performance of the 
overwhelming majority of federal social programs. Faced with this lack of knowledge about the 
effectiveness of federal social programs, it is past time for Congress to devote serious attention 
and resources to finding out what works and does not work 

Can Effective Programs Be Replicated? 

Policymakers and advocates often assume that a social program tliat is effective in one 
setting will automatically produce the same results in other settings. Some proponents of 
evidence-based policy even make this faulty assumption. For example, adv'ocates of expanding 
early childhood education programs make scientificali}^ unsupportable generalizations regarding 
effecth eness based on two small-scale e\ aluations — the High 'Scope Perry Preschool and 
Carolina Abecedarian Projects"'"' — ^that are no^vhere near being the definith'e studies on the 
subject.""* Policymakers should be very skeptical about the speculated paj'offs of implementing 
these programs on a national scale. Tlie evaluation of the Perrj'^ progi’am began in 1962. Despite 
all the hoopla, the results have never been replicated. In more them 50 years, not a single 
experimental evaluation of the Pei'ry approach applied in another setting or on a larger’ scale has 
produced the same results. The same holds tme for the Abecedarian program, wkich began in 
1972. Tliere is no evidence that these programs can produce the same results today. 

Many advocates of social pmgrams have adopted the language of the “evidence-based’’ 
policy movement. Under the evidence-based policy movement, programs found to be effective 
using rigorous scientific methods are deemed “effective” or “evidence-based” and held up as 
“model’'' programs. Tlie assumption is that the same successful impacts found at a particular 
setting can be replicated in other settings or on the national scale. 

However, many of the programs labeled as “evidence-based” — often by program 
advocates — ^have been evaluated in only a single setting, so the results cannot necessarily be 
generalized to other settings, hi addition, these evidence-based programs have often been 
implemented by highly trained professionals operating under ideal conditions. In the real world, 
program conditions are oft;en much less than optimal. For example, based upon the results of the 
Abecedarian Project, Congress created Early Head Start — a national program that serves lotv- 
income f^nilies wdth pregnant women, infants, and toddlers up to age three.^^ Howevei', the 
results of a multisite experimental evaluation of the national program found few initial modest 
impacts that quickly faded way."''* 

Another excellent example of the federal government replicating an effective local 
program is the Center for Employment Training (CET) RqDlication." Of 13 youth job-training 
programs ev aluated, the JOBSTART Demonstration found only one program to have a positiv’^e 
impact on earnings — the Center for Employment Training (CET) in San Jose, California.^ Based 
on the results for the CET, the U.S. Department of Labor replicated and evaluated tlie impact of 
CET ill 12 other sites.^*^ The CET model had little to no effect on short-lemi and long-term 
cmpio^micnt and earnings outcomes at these other locations. The multisite experimental 
evaluation of CET, according to its authors, “shows, that ei^en in sites that best implemented the 
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model, CET had no o^^erall employment and earnings effects for youth in the progi’am, even 
though it increased participants' hours of training and receipt of credentials.”'’ 

Just because an innovative program appears to have worked in one location, does not 
mean that the program can be effectively implemented on a larger scale. 

What Congress Should Do 

Congress needs to take the lead in making sure that the social programs it funds are 
evaluated. First, when authorizing a new social program or reauthorizing an existing program. 
Congi'ess should specifically mandate multisite experimental evaluation of the program. 
Congressional mandates are nece.ssary because federal agencies often resist performing 
experimental evaluations. For example, many jurisdictions receiving ftinding through the Job 
Training Partnership Act (JTPA) and Job Opportunities and Basic Skills (JOBS) programs 
refiised to coopei ate with large-scale expei imentai evaluations of these programs.^^ 

Experimental evaluations aie tlie only way to determine to a high degree of certainty the 
effectiveness of social programs. Thus, Congress should mandate that all recipients of federal 
funding, if selected for participation, must cooperate with evaluations in order to receive future 
fimding. 

Second, the experimental evaluations should be lai’ge-scale, nationally representative, 
multisite studies. When Congress creates social programs, tlie funded activities are intended to 
be spread out across the nation. For this reason. Congress should require nationally 
representative, multisite experimental evaluations of these programs. For miiltisite evaluations, 
the selection of the sites to be evaluated should be representative of tlie population of interest for 
the program. When program sites and sample participants are randomly selected, the resulting 
evaluation findings will have high external validity. 

While individual programs funded by federal grants may undei'go experimental 
evaluations, these small-scale, single-site evaluations do not inform policymakers of the genei'al 
effectiveness of national programs. The success of a single program that sers'es a particular 
jurisdiction or population does not necessarily mean that the same program will achieve similar 
success in other jurisdictions or among different populations. Thus, small-scale evaluations are 
poor substitutes for large-scale evaluations. In addition, a multisite experimental evaluation that 
examines the perfomiance of a particular program in numerous and diverse settings can 
potentially produce results that are more persuasive to policymakers than results firoin a single 
locality. 

The Building Strong Families (BSF) demonstration project sponsored by the I J.S. 
Department of Health and Human Services is an excellent example of a program that had 
varying impacts by location. BSF provided counseling services to unmairied couples w'ho were 
expecting or had recently had a baby in eight sites. The marriage progi'ani’s intent was to steer 
low-income unmarried couples with or expecting a child tow^d marriage. 

The eight-site demonstration project underwent an experimental evaluation that reported 
findings for 1 5- and ?6-month follow-up periods. The 36-month follow-up study concluded; 
“After three years BSF had no effect on the quality of couple’s relationships and did not make 
couples more likely to stay together or get married.”^' In addition at the 36-moiith follow-up 
period, “BSF had no effect on couples’ co-parenting relationship; it had small negative effects on 
some aspects of fallier involvement.”^'^ Not to be dismissed, the long-term folioiv-up did find a 
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beneficial impact of increased socio-emotionai dc‘\'clopmcnt foi’ children in the interv^ention 
group, compared to children in the control group. 

While the evaluation of the eight demonstration sites found federally funded marriage 
promotion programs to be ineffective overall, the results from Atlanta, Baltimore, Oklahoma 
City, and the Florida counties were contradictory In Atlanta, RSF led to a long-term decrease 
in the ability of participants to avoid destructive corrflict behaviors. In Baltimore, urmiamed 
couples participating in the program were less likely to be still romantically involved at dre time 
of the i5-month foilow-up. In addition, couples in the Baltimore program rqaortcd less support 
and affection in their relationships, and fathers were less likely to provide financial support for 
their children and less likely to engage in cognitive and social play with their children. By the 
time of the 36-month follow-up, these harmful impacts in Baltimore faded away. 

Wliile the short-term findings for the Florida counties indicated that the BSF yielded no 
beneficial or harmful impacts on participants, the long-term findings indicate the presence of 
several haimfiil impacts. For the relationship status of the couples, intervention group couples 
w'crc less likely to be romantically involved and living together (manded or unmanied), 
compared to their counterparts in the control group. In addition, fathers in the intervention group 
w'ere less likely to live with and regularly spend time with their child. 

In Oklahoma City, the opposite occun'ed. While umnarried couples in the program were 
no more likely to many than were the control group couples at the time of the 15-month follo^v- 
up, Oklahoma participants reported improvements in relationship happiness, support and 
affection, use of constructive conflict behaviors, and avoidance of destriiclive conflict behaviors. 
AdcUtionally, fathers participating in the program "were more likely to provide financial support 
for their cliildren than ^vere their coiuiterparts in the control group. While BSF still had no effect 
on marriage rates at tlie time of the 36-month foilow-up, couples in tlie intervention gi'oup were 
more likely to rejxirt that neither partner had been unfaithful since random assignment, compared 
to control group couples. 

If the Atlanta, Baltimore, and Florida counties sites were the only sites evaluated, then 
the results would indicate that federally sponsored marriage coimseiing for unmarried couples 
wifo children has hamifui effects. Relying only on the more positive Oklahoma City results 
w'ouid have led to the opposite conclusion. 

Contradictory results from evaluations of similar social programs implemented in 
different settings are a product not only of implementation fidelity.'^ but also of the enormous 
complexity of the social context in which these programs are implemented. .Tim Manzi, a senior 
fellow at the Manhattan Institute, uses the conflicting results of experimental evaluations to 
explain the influence of “causal density" on the social sciences. ''^ “Causal density,” a term coined 
by Manzi, i.s “the number and complexity of potential causes of the outcomes of interest.”'^ 
Manzi postulates that as causal density rises, social scientists will find greater difficulty in 
identifying all of the factors that cause the outcome of interest. 

The confounding influence of causal density, in addition to implementation iideiity, 
likely contributed to contradictory effects of federal marriage promotion progmins by location. 

To address causal density, expenmcntal impact evaluations of fecbral social programs should be 
conducted using multiple sites. In fact, the total sum of the multiple sites should be nationally 
representative of the populations seived by the social program being ev^aluated. 
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Causing Harm 

The results of the 20 multisite experimental evaluations of 2 1 federal social programs 
published since 1990 generally find that these programs are ineffective?^ Howev er, social 
program advocates too frequently concentrate on any beneficial, even if only modest, impacts 
that have been identified. Nevertheless, politicians and policy experts also need to recognize that 
federal social programs can produce hannful impacts too. These harmful effects rarely get 
mentioned in government press releases announcing the findings of evaluations. In addition to 
the BSF findings, die following is a brief summary of the harmful impacts found in miiltisite 
experimental evaluations of federal social programs published since 1990.^^ 

For Early Head Start, white parent.s in the intervention group displayed liigher 
dysfunctional parent-child interactions than their counterparts in the control group.'^^' Further, 
participation in Early Head Start appears to have increased welfare dependency for Hispanics. 

Enhanced Early Head Start with Employment Services is a demonstration program diat 
involves regular Early Head Start services wdth the addition of employment and training services 
for parents. An experimental evaluation of the program based on t\^'o sites in Kansas and 
Missouri was performed. At the time of the 4S-month follow-up, the longest job spells of 
mothers participating in the program were significantly shorter than the job spells ol' mothers in 
the control group."*^ 

For the tliree-year-old cohort of the Head Start Impact Study, kindergarten teachers 
reported that math abilities were worse than for similar children not given access to the 
program."^^ For the four-year-oid cohort, teachers reported tliat Head Start children in the first 
grade were more likely to be shy or socially reticent than their peers. By the third grade, teachers 
reported that the four-year-old cohort with access to Head Start displayed a higher degree of 
unfavorable emotional symptoms than similar children without access to the program.^' Fiuther. 
children in tire four-year-old cohort self-reported poorer peer relations with fellow children than 
their counterparts in the control group.'*'* 

The role of the federal government in funding after-school programs increased 
substantially after passage of the Improving America’s School Act of 1994, which created the 
2 1st Century Conmiunity Learning Centers program. A multisite expeitmental impact evaluation 
of the 21st Century Community Learning Centers program fomid a whole host of harmful 
effects. Overall, teacher's found participating students to have disciplinary^ problems that were 
confirmed by student-reported data. According to their teachers, participating students were less 
likely to achieve at above average or high levels in class and were less likely to put effort into 
rcading or English classes. These students were also more likely^ to have behavior problems in 
school than their counterparts. Teachers were more likely to have to call the parents of 
participating students about misbeha\'ior. Participating students were also more likely to miss 
recess or be placed in the hall for disciplinary reasons, wliile also having parents come to school 
more often to address behavior problems. 21st Century' students were also more likely to be 
suspended from school than similar students, 

Upwaivl Round was created in 1965 and is an original War on Poverty social program. 
Through the provision of supplemental academic and support services and activities, Upw'ard 
Bound is intended to help economically disadvantaged high school students successfully 
complete high school and attend college. Despite the program’s lofty' goal, Upward Bound 
participants with high expectations to earn a college degree were less likely than their 
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counteiparts to earn associate’s degrees, while being no more or less iikety to attain any other 
college degree 

The Departoient of Health and Human Ser\'ices and Department of Labor funded tlie 
Employment Retention and Advancement (ERA) project, initiated in 1 998, to assess the 
effectiveness of 12 different employment retention and advancement programs across the 
nation.*^ ' Participation in ERA programs targeting unemployed Temporary Assistance for Needy 
Families (TANF) recipients in Houston, Texas, and Salem, Oregon, was associated with 
increased dependence on the receipt of TANF benefits, while participation in the program in Fort 
Worth, Texas, was associated with increased dependence on food stamps. The Chicago ERA 
program targeting employed TANF recipients was associated with increased dependence on food 
stamps, while the Medford. Oregon, ERA program targeting employed individuals not on TANF 
was associated with decreased employment. 

Conducted in five cities, the Moving to Opportunity (MTO) demonstration assessed the 
impact of offering families with children under 18 living in public housing developments or 
concentrated poverty'^ areas the opportunity to move out of their neighborhoods. The evaluation 
consisted of tw'o intervention groups, MTO voucher recipients and Section 8 voucher recipients, 
compared to a control group that did not receive MT(.) or Section 8 vouchers but was eligible to 
receive public housing assistance. For adults and children widi access to MTO or Section 8 
vouchers, several hannful impacts were produced."*^ Access to a MTO voucher was associated 
with increased dqiendence on dmgs and alcohol for adults. Also, MTO adults had higher 
participation rates in food stamps and received more food stamp benefits than their similar 
counteiparts not given access to MTO or Section 8 vouchers. Youth from families given access 
to MTO vouchers were less likely to be employed and more likely to have smoked than their 
peers. These youth were also more likely to be airested for property cimies. As for Section 8, 
adults offered access were more likely to be currently unemployed and less likely to have 
employment spells with the same job for at least a year. In addition. Section 8 adults were less 
likely to be currently working and not receiving TANF than their counterparts. Section 8 youtli 
were more likely to have smoked than their peers in the conhol group. 

Adult men participating in J TPA programs w-ere more likely to be dependent on AFDC 
benefits than similar men not given access to the training.'^® Male y^ouths with no criminal arrest 
record at the time of random assignment weie more likely to be arrested after participating in 
federal job-training programs, while male youth with histories of arrest experienced long-term 
declines in income. 

Ill an attempt to help Americans start businesses, the Department of Labor teamed with 
the Small Business Administration to create an emplo^nent program to assistant people in 
creating or expanding their own business enterprises. After receiving entrepreneurship training. 
Project GATE participants spent more time collecting Unemployment Insurance benefits than 
their counterparts that were not taught how to be entrepreneurs. UTiile Project GATE had no 
effect on the sclf-cmployment income of participants, participants experienced initial periods of 
decreased wages and salaries earned from ov erall employment. 

The Quantum Opportunity- Progium (QOP) demonstration, operated by the I.I.S. 
Dcpai'tmcnt of Labor and the Ford Foundation from 1995 to 2001, offered intensive and 
comprehensive services with the intention of helping at-ri.sk youth gr aduate from high school and 
enroll in postsecondary education or training. Q(^P provided services to participants year-round 
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for five 5 'ears. The findings from the QOP experimentai ei'aluation, according to its authors, 
provide some insight about the effectiveness of WIA youth programs. For the initial post- 
interv'ention impacts, youth participating in QOP were less likely to find jobs that provided 
health insurance benefits."’ At the six-year follow-up period, youth participating in QOP were 
more likely to be airested.^^ Increasing criminality appears to be a common effect of federal job- 
training programs supposedly benefiting youth. 

The previously discussed CET Replication job-tra inin g programs were associated with 
several harmftil outcomes. Men experienced periods of declines in employment, earning.s, and 
number of months worked. Individual participants who possessed a high school diploma or GED 
at the time of random assignment experienced periods of declines in the number of months 
worked and earnings. In addition, participants in the high-fidelity sites were less likely to find 
jobs that provided health insurance. x\lso, those older than 18 and those with high school degrees 
or GEDs at the time of random assigimient were less likely to have jobs that provided health 
insurance. 

Job Corps is another federal training program that has negative effects. Created in 1964, 
Job Corps is a residential job-training program tiiat serves disadvantaged youths aged 16 to 24 in 
125 sites across the nation. A multisite experimental evaluation of Job Corps found, compared to 
non-participtaits, Job Corp participants were less likely to earn a high school diploma. In 
addition, youth participating in the program worked fewer weeks and worked fewer hours per 
w'eek than similar youth in the control group."' 

In sum, federal social programs that hann their participants are not unco mm on. This fact 
is all too often ignored by advocates of these social programs. 

Conclusion 

With the federal debt reaching staggering heights, Congi'ess needs to ensure that it is 
spending taxpayer dollars wisely. Muitisitc experimental evaluations arc the best method for 
assessing the effectiveness of federal social programs. Yet to date, this method has been used on 
only a handful of federal social programs. While preGous results have been disappointing. 
Congress needs to reverse the trend of not rigorously evaluating federal social programs. 

The Heritage Foundation is a public policy, re.search, and educational organization recognized as 
exempt under section 501(c)(3) of the Internal Revenue Code. It is privately supported and 
receives no funds from any government at any level, nor does it perform any government or 
other contract work. 

The Heritage Foundation is the most broadly supported think tank in the United States. During 
2012, it had nearly 700,000 individual, foundation, and corporate supporters representing eveiy 
state in the U.S. Its 2012 income came from the following sources; 


Individuals 

81% 

Foundations 

14% 

Corporations 

5% 


rhe top five corporate givers provided I he Heritage Foundation with 2% of its 2012 income. 
The Heritage Foundation’s books are audited annually by the national accoimting finn of 
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McGladrey & Pullen. A list of major donor’s is available from The Heritage Foundation upon 
request. 

Members of The Heritage Foundation staff testify as individuals discussing their own 
independent research. The views expressed arc their own and do not reflect an institirtional 
position for The Heritage Foundation or its board of hustees. 
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Chairman REICHERT. Ms. Smith, you are recognized for 5 min- 
utes. 

STATEMENT OF TARA SMITH, RESEARCH ASSOCIATE, RAY 
MARSHALL CENTER, LYNDON B. JOHNSON SCHOOL OF PUB- 
LIC AFFAIRS, THE UNIVERSITY OF TEXAS 

Ms. SMITH. Thank you. Good afternoon, Chairman Reichert, 
Ranking Member Doggett, and Members of the committee. My 
name is Tara Smith, I am with the Ray Marshall Center for the 
Study of Human Resources at the University of Texas at Austin’s 
LBJ School of Public Affairs. Thank you for inviting me today. 
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This hearing’s focus on families coincides with a growing body of 
research on two-generation programs designed to link services for 
children and parents so that families as a whole can build the 
human capital they need to succeed in school and the labor market. 
Today, I would like to share findings and lessons learned from 
evaluations of two such programs. 

Capital IDEA is a sectoral-based training program in Austin, 
Texas, that was built on a model pioneered by Project QUEST in 
San Antonio for employer-driven work force development. Capital 
IDEA provides training primarily in health care for low-income and 
disadvantaged adults. The evaluation tracks participants from 
2003 forward and includes outcome, impact, and ROI analyses. 
CareerAdvance is a career pathways program in Tulsa, Oklahoma, 
which provides parents of Head Start and early Head Start stu- 
dents with training for health care occupations. This program 
launched in 2009, and the evaluation includes an implementation 
study as well as outcome and impact studies focused on parents 
and children. 

The impact evaluations for both programs use quasi-experi- 
mental research methods based on a carefully matched comparison 
group. There are five key points I would like to emphasize about 
these evaluations. First, rigorous quasi-experimental methods have 
been found to produce impact estimates similar to those found in 
random control trials. Quasi-experimental methods also address 
issues such as the localized nature of programs which limits the 
pool of prospective and eligible applicants needed to support a ran- 
dom control trial. 

Second, the use of administrative records and propensity score 
matching techniques helps to keep evaluation costs reasonable. In 
both evaluations State UI records provide consistent, comprehen- 
sive, and inexpensive data on employment and earnings. For Cap- 
ital IDEA, the comparison group is drawn from individuals who re- 
ceive job search assistance at a local one-stop career center and 
who closely resemble participants along 18 characteristics, includ- 
ing demographics and prior employment and earnings history. For 
CareerAdvance, the comparison group is drawn from other Head 
Start parents matched along multiple characteristics, including a 
documented interest in pursuing further education and training. 
The rigor of the comparison group matching design undergirds our 
confidence in the evaluation findings. 

Third, sectoral and career pathway models have demonstrated ef- 
fectiveness in a number of industries and labor markets by con- 
necting low-income and low-skilled adults with the training they 
need to enter higher paying careers. If you have my written testi- 
mony in front of you, the chart on page 3 shows our most recent 
findings from Capital IDEA. We find that on average the earnings 
of participants continue to grow over time while comparison group 
Members who receive only job search assistance or other basic 
work force services tend to have relatively fiat earnings. 

My fourth point is that two-generation strategies which look to 
build on sectoral or career pathway programs by linking those 
adult education and work force training services with high quality 
educational opportunities for children show promise. Wrap-around 
and family support services including child care, transportation as- 
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sistance, counseling, and other resources ensure that participants 
in both generations receive the help they need to achieve at a high 
level. 

In CareerAdvance this support can include a monthly financial 
incentive for performance and attendance to help offset the costs of 
participation and provide some financial stability for the family. 

Finally, because social programs rarely involve cookie cutter ap- 
proaches, it is important to consider a broad base of evidence when 
evaluating program effectiveness. Implementation and process 
evaluations provide important context for understanding how pro- 
grams operate and identifying which services may lead to better 
outcomes and impacts over time. This is particularly true for new 
and emerging programs and program replication efforts. 

In conclusion, strategies that focus on basic skills which provide 
counseling and other support services, which increase opportunities 
to earn and learn so that parents can support their families while 
in training and target skill development at high wage, high de- 
mand occupations in the local labor market all appear to have sig- 
nificant rigorous evaluation support and could be promoted in Fed- 
eral programs. 

By investing in proven approaches and promising strategies, 
such as two-generation initiatives, and supporting a broad range of 
research and evaluation efforts on those investments, the Federal 
Government can play an integral role in building the knowledge 
base needed to expand and improve efforts to move families out of 
poverty. Thank you. 

Chairman REICHERT. Thank you, Ms. Smith. 

[The prepared statement of Ms. Smith follows:] 
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Testimony before the Subcommittee on Human Resources, Committee on Ways 
and Means, U.S. House of Representatives 

Tara Smith, Research Associate, Ray Marshal! Center for the Study of Human Resources at 
The University of Texas at Austin's Lyndon B. Johnson School of Public Affairs 

July 17, 2013 


Good afternoon Chairman Reichert, Ranking Member Doggett, and Members of the 
Subcommittee. My name is Tara Smith; i am a Research Associate at the Ray Marshall Center 
for the Study of Human Resources, which is part of The University of Texas at Austin's Lyndon B. 
Johnson School of Public Affairs. Thank you for inviting me to testify about evaluation efforts to 
help families support their children and escape poverty. The two-generation focus of this 
hearing coincides with a growing body of research and demonstration programs that seek to 
link services for children and parents so that families as a whole can build the human capital 
they need to succeed in school and the labor market. 

Today i would like to share with you the findings from the Ray Marshal! Center's evaluation of 
two sector-based training programs with an identified two-generation focus {Sommer et ai, 
2011). The first. Capita! IDEA, is a program that operates in Travis County, Texas. The Center's 
evaluation tracks participants over time, starting with the cohort that began in 2003. The 
second program is CareerAdvo/Jce®, a program launched by the Community Action Project of 
Tulsa County, Oklahoma (CAP) in 2009. Both evaluations incorporate a rigorous quasi- 
experimental design to 1) address issues related to the localized nature and selectivity of the 
programs; and 2) work within funding constraints by taking advantage of inexpensive 
administrative data sources and propensity score matching techniques, in both cases the 
available pool of eligible and interested applicants is not iarge enough to support a random 
control trial. The programs have developed selective enrollment requirements to target 
resources at individuals who are better prepared to enter and succeed in intensive skills 
training. Key data sources available tor both evaluations include program progress data and 
state unemployment insurance (Ul), workforce, and other public benefit records. 

Evaluation of Capital IDEA 

Since 2006, my colleagues at the Ray Marshall Center and i have been conducting an outcomes 
and impacts evaluation of local workforce development investments.^ Travis County, Texas, 
home to state capital Austin, is fairly unique among local governments: each year it invests 
approximately $1.5 - 2 million local tax dollars in job training and other workforce services for 
disadvantaged residents, including individuals who are at or below 200 percent of the poverty 

^ All of the research reports from this evaluation is available at; www.raymarshallcer, ter.org 
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line, ex-offenders, and individuals without high school education credentials,^ Perhaps even 
more unique, Travis County funds the Ray Marshal! Center to conduct a rigorous ongoing 
evaluation of those investments. 

Capital IDEA is one of the programs whose evaluation is funded by Travis County. Annual 
reports document participant outcomes across four measures over time: quarterly 
employment, quarterly earnings of those employed, claims filed for Ut benefits, and monetary 
eligibility for Ui benefits in the event of a job loss— a proxy measure for employment stability 
given that, to be eligible, Individuals must have earnings in the first four of the last five quarters 
prior to filing a U! claim. 

The impacts analysis follows a quasi-experimental methodology with the creation of a carefully 
matched comparison group drawn from individuals who received job search assistance or other 
universal services at local One-Stop Career Centers, individuals are matched along at least 18 
dimensions, including demographics and prior employment and earnings, impacts are 
measured by statisticaliy significant differences between outcomes from participation in Capita! 
IDEA versus outcomes from accessing other job search assistance services in the community. 
This methodology follows research by Greenberg at a!. (2006), Hollenbeck and Huang (2006), 
Card et al. (2010), and Heinrich et al. (2010), all of which have found that such quasi- 
experimental designs can produce impact estimates comparable to those resulting from more 
rigorous and costly approaches, I.e,, randomized control trials. As recently advocated by 
Maureen Conway and her colleagues at the Aspen Institute (2012), state UI (and other state 
workforce) records provide a key advantage given the modest Travis County evaluation budget: 
it is relatively inexpensive to collect the data and run the analysis. 

Capital IDEA takes a sectoral approach through strong collaborations with Austin Community 
College and employers and industry groups to prepare participants to succeed in the workplace. 
Capital IDEA supports long-term training primarily in healthcare (approximately 75 percent of 
participants), including nursing and allied health occupations such as dental hygienist, surgical 
technician, and occupational or physical therapy assistant, Training is also available in 
technology careers and professional trades currently paying a starting wage of $16 or more per 
hour with benefits. The range of training options supported by Capital IDEA has changed over 
time based on the needs of the Austin labor market and prevailing occupational wages. 

The most recent impact evaluation Includes 879 Capital IDEA participants who started and 
either completed or dropped out of the program between 2003 and 2008, and examines 
outcomes through March 2011 (Smith, King, and Schroeder, 2012). Capita! IDEA participants 


^ It should be noted that the City of Austin also invests a similar amount of its local tax dollars each year in 
workforce development services for disadvantaged residents. 
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significantly out-performed the comparison group on three measures. Across all post-service 
quarters, participants had a 12.3 percentage point advantage over the comparison group in the 
share employed as wet! as the share with sufficient employment and earnings to meet the 
monetary eligibility standards for Ul benefits. Employed Capital IDEA participants also earned 
an average $759 more in each post-service quarter than the comparison group, at an average of 
$7,104 per quarter over the post-service period. Unemployment claims were low {under 3 
percent) and not significantly different between the groups. 

The evaluation also compares the unconditional earnings over time of participants and 
comparison group members to illustrate the relationship between quarterly employment and 
earnings. Unconditional earnings are averaged across ail of the individuals in a group regardless 
of employment status. After having very similar earnings in the eight quarters prior to seeking 
workforce development services— demonstrating the quality of the comparison group 
matching process— participants entering Capital IDEA experienced a trough of earnings In 
relation to the comparison group for several quarters (Figure 1 below). Capital IDEA 
participants are typically in training for eighteen months and often only work part-time during 
that period. Starting in the sixth quarter after service, however, participants out-earned the 
comparison group In each subsequent quarter examined. While the comparison group's 
earnings have been relatively flat, the earnings of Capita! IDEA participants have continued to 
climb. The earliest cohorts earned approximately $2,000 more each quarter than the 
comparison group in five and one-half to seven years after program entry. 


Figure 1: Capital IDEA Participant Earnings Over Time Versus Matched Comparison Group 
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The design of Capita! IDEA grew out of a mode! pioneered by Project QUEST in San Antonio, 
Texas, for employer-driven, intensive skil! development with a focus on investing in people 
(Deaton and McPherson, 1991; Campbeli, 1994}. As a sectoral program. Capital IDEA 
collaborates with employers and training providers to help prepare participants to succeed in 
available jobs. The program regularly reviews training offerings and local labor demand with 
employers and industry groups to shape program offerings to meet the needs of the local 
economy. A close collaboration with Austin Community College, the provider for the majority 
of the occupational training programs, has also resulted in Capita! IDEA'S College Prep 
Academy, which prepares individuals with at least 5^'’ grade skills to enter college-leve! courses 
rather than traditional developmental education. 

it is important to note that Capita! IDEA has a selective enrollment process that is intended to 
identify individuals who are motivated and capable to succeed in an intensive program. Capita! 
IDEA covers a!! of the costs associated with the training, and provides or funds multiple support 
services designed to help individuals achieve their employment goals. Child care and 
transportation assistance are key supports for many participants who would otherwise not be 
able to afford to spend time in training. Weekly participant VIP (Vision, Initiative, and 
Perseverance) group meetings and individual conversations with career counselors help keep 
participants focused on the opportunity and build the critical soft skills in communication, time 
management, and financial literacy that will better prepare them for the workplace. 

In 2011, the Ray Marshal! Center conducted an exploratory return on investment (ROi) analysis 
of the program (Smith and King, 2011). Capital IDEA is funded through a mix of public and 
private resources. Approximately two-thirds of the program's budget is derived from 
taxpayers, while the remaining one-third is funded by employers, alumni, individuals and 
philanthropic organizations. Using the impact evaluation findings for the 2003-2004 cohorts of 
Capital IDEA participants, we completed a benefit-cost analysis based on actual program 
expenditures (which averaged approximately $6,500 per participant), foregone earnings, and 
outcomes using conservative assumptions for fringe benefits, future earnings growth, the 
discount rate, taxes, and public benefits. We found that the taxpayer's investment was fully 
recouped after 8.5 years. Over the first ten years, each dollar invested in Capital IDEA was 
estimated to return $1.65 to taxpayers — an annual rate of return of approximately 9 percent. 
Over 20 years that rate rises to 17 percent with each taxpayer dollar in Capital IDEA returning 
$5.01. 

The Ray Marshall Center is not the only organization evaluating Capital IDEA. Researchers for 
the Aspen institute's Workforce Strategies Initiative have studied the Capital IDEA-Austin 
Community College collaboration along with five other sector-based nonprofit-community 
college partnerships across the country to identify how such partnerships provide the supports 
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and services that low-income learners need to succeed In training and enter employment. The 
Courses to Employment evaluation seeks to identify effective partnership models and key 
program components that lead to better student outcomes. The most recent update from the 
study (Helmer, 2013) examines how each of the partnerships helps adult learners navigate 
community college and the labor market. College entry, persistence and completion, and the 
transition to the labor market are the three major challenges facing many adult learners. The 
research highlights Capita! IDEA’S College Prep Academy, the program's peer support group 
meetings and career counselors, and Its ongoing case management and alumni network as 
promising strategies for addressing those common challenges. 

Finally, recent indicative evidence on Capital IDEA includes a survey of seventy-eight graduates 
to gather information about intergenerationa! impacts (McCollum, 2011). Almost ninety 
percent of respondents' children who are now 18 or older have received a high school diploma, 
while another 7 percent have received a GED. More than half (53 percent) of those children are 
currently enrolled in college while 11 percent have already earned a college degree. An 
additional 11 percent had some college experience but were not currently enrolled at the time 
of the survey. The survey respondents strongly connected their participation in Capital IDEA 
with their children's academic performance and their expectations for their children to attend 
college. 

Given this full body of evidence, the Capital IDEA sectoral/career pathway mode! appears to be 
a successful strategy for helping to lift families out of poverty. The quasi-experlmental 
methods, combined with multiple evaluation sources, appear to demonstrate significant 
support for the conclusions drawn about program effectiveness. Further tests of the mode! in 
different labor markets would help to identify challenges and alternative approaches for 
successful replication. Recent Federal investments, including the U.S. Department of Health 
and Human Services-funded Innovative Strategies Initiative for Self-Sufficiency (ISIS) and the 
U.S. Department of Labor's Workforce Innovation Funds, should provide important evidence on 
these topics. 

Evaluation of CareerAdvance® 

Through my work at the Ray Marshal! Center, I have also been involved in the development and 
evaluation of a two-generation strategy in Tulsa, Oklahoma. Career Advance® provides training 
for parents of children in Head Start or Early Head Start, including TANF recipients, for careers 
in nursing, health information technology, and other allied health professions. CareerAdvance® 
was launched In 2009 based on a program design created by the Ray Marshall Center in 
collaboration with researchers at Harvard University and the Community Action Project of Tulsa 
County (CAP) which, among other activities, runs most of Tulsa's Head Start and Early Head 
Start centers (King et ai., 2009). CAP contracts with Tulsa Community College and the Tulsa 
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Technology Center to provide training along several healthcare career pathways. Other key 
components of the program include career coaching, peer support groups, childcare assistance, 
and monthly financial incentives based on performance and attendance. 

in Tulsa, a multi-method evaluation is underway to determine whether this sectoral/career 
pathway training program linked to high quality early childhood education programs and other 
supports will result in better outcomes for both parents and children over time. Originally 
funded by the George Kaiser Family Foundation, the program is now supported by the 
Administration for Children and Families' Health Professions Opportunities Grant (HPOGj 
program. The CAP Family Life Study evaluation is funded by the HPOG-University Partnership 
initiative and the W.K. Kellogg Foundation, and ied by P. Lindsay Chase-Lansdale at 
Northwestern University and Christopher T. King, Director at the Ray Marshall Center. The 
comprehensive evaluation design includes qualitative and quantitative research components to 
understand educational and labor market impacts for parents who participate in 
CareerAc/vonce® and their children (Chase-Lansdale and King, 2013). 

The variety and breadth of the data assembled for the evaluation are particularly appropriate 
for a program at this stage in its development. CareerAdvonce® is still refining its service and 
training approach through a regular feedback loop among evaluators, program staff, training 
providers, and Tulsa healthcare industry employers. This focus on continuous improvement is 
one factor that seems to be tied to program effectiveness. A key piece of evidence for program 
improvement is the implementation evaluation of CareerAc/vance® ied by the Ray Marshall 
Center. I draw your attention to this research to underscore how important it is to fund more 
than outcomes and Impact studies when evaluating social programs, especially new and 
emerging ones. The implementation study provides important context for interpreting 
CareerAdvonce® outcomes and impact findings, implementation research and process 
evaluations are needed, particularly in pilot and demonstration projects, to discover how a 
planned program design operates in reality, to document challenges and any program 
responses over time, and to gain insight from key stakeholders to identify lessons learned and 
recommendations for other partnerships that may be interested in replicating or adapting the 
approach in their community (Smith, Douglas, and Glover, 2012). 

The quasi-experimental impacts analysis underway in Tulsa uses a carefully matched 
comparison group drawn from parents of children in CAP early childhood centers who do not 
participate in CareerAdvonce®, but who have shown significant motivation for and interest in 
further education and career training in surveys conducted by Family Support workers, in the 
impacts study, participant and matched comparison group parents alt complete regular surveys 
and interviews and consent to have their employment and public benefit records released for 
analysis. Further, the research identifies a target child in each family to follow In their 
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transition from CAP'S early childhood centers to elementary school and beyond to document 
second-generation outcomes. Regular assessments with the child; based on measures that are 
validated in the evaluation literature, as well as surveys of the child's teachers provide 
information important to determining how a parent's participation in job training impacts the 
child's educational experiences. This research is in year three of a funded five-year study 
(Chase-Lansdale, etal., 2013). 

Other Evidence in Support of Two-Generation and Sector Strategies 

! believe the Capital IDEA and CareerAc/vonce® evaluations contribute important evidence to 
the growing research base in support of sector-based or career pathway workforce 
development and two-generation strategies designed to lift low-income adults and their 
families out of poverty. Other researchers have found that social programs, particularly 
education and training programs focused on improving longer-term economic outcomes, also 
can have a significant effect on families. 

In Tuning into Local Labor Markets: Findings from the Sectoral Employment impact Study, 
Maguire et al. (2010) report on the results of an experimental study of three sector-based 
programs.^ The findings reveal strong impacts, including a 32 percentage point increase in 
training participation, and an 18 percent increase in earnings over a two-year period. The 
authors conclude that "the study provides compelling evidence that nonprofit-ied sector- 
focused training programs can increase the earnings of a range of disadvantaged populations" 
(page 54). 

in a study for the Annie E. Casey Foundation, Magnuson (2007) concluded that "evidence 
suggest that parents who succeed in completing additional schooling or secure a higher-paying 
job are likely to have children with better health, schooling, and labor market outcomes" (p. 7). 
One promising approach is Washington State's i-BEST Model (integrated Basic Education and 
Skills Training Program). The i-BEST Model has gained national attention through evaluations 
showing that it is a highly effective strategy for low-income adults who need to quickly develop 
basic and occupational skills for fast entry into the labor market (Zeidenberg et ai., 2010). Using 
a quasi-experimental methodology, evaluators have found a 17 percentage point increase in 
service participation and a 7.5 percentage point increase in the rate of occupational 
certifications earned within three years of initial enrollment, This strategy has important 
implications for helping low-income adults overcome poor basic academic skills and avoid the 
trap of developmental education which too often prevents individuals from pursuing college- 
level training for higher-wage occupations. 


^ Per Scholas in New York City, Jewish Vocational Services in Boston, and the Wisconsin Regional Training 
Partnership in Milwaukee. 


7 



54 


In a review of the evaluation literature, King and Heinrich (2011) found considerable evidence 
that workforce development investments are effective and can produce significant returns for 
participants, employers, taxpayers, and society. They note sector-based programs in particular 
as a promising strategy for preparing low-income adults for opportunities in the labor market. 
The authors sound a warning, however, on declining investments in workforce development at 
a time when the nation is stiii struggling to recover from the Great Recession. Younger workers 
and low-skilled, iow-income adults in particular need proven, targeted interventions to prepare 
for, obtain, and persist in employment. 

Concluding Thoughts 

Sector-based strategies and related career pathway programs which are targeted at local labor 
market needs and which support low-income adults in overcoming barriers to program 
completion and labor market transitions have demonstrated effectiveness in a number of 
communities and industries (Glover and King, 2010). Tying those strategies, which are targeted 
at adults, with services and high quality educational opportunities for children is the logical next 
step (Smith, 2012; Ascend, 2012). Thoughtfully connected programs that target whole families 
for advancement have demonstrated some promising preliminary results. While the evidence 
base in support of two-generation strategies is currently being built, there is much the Federal 
government can do to act on the available evidence. 

Federal programs, whether contract- or grant-funded, should encourage or require investments 
in program models and strategies that have consistently been found to be effective. Strategies 
that focus on basic skills, provide counseling and other support services, increase opportunities 
to “earn and learn" so that parents can support their families while in training, and target skill 
development at high-wage, high-demand occupations in the local labor market all appear to 
have significant, rigorous evaluation support and could be promoted in Federal programs. 
Because no single approach would work in every community or with every disadvantaged 
population, it is important to identify which strategies appear to be best applied in a given 
context The Federal government can help build the needed knowledge base for understanding 
what works by funding evaluation studies of its investments using a variety of research 
methods that have been proven both rigorous and cost-effective. 


Thank you for your attention; ! would be happy to an-swer any questions you may have about 
my work or the programs and research I have discussed here today. 
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Chairman REICHERT. And for your information, panelists, now 
is the question-and-answer phase, so I am sure the Members on 
the Committee would like to ask a few questions. I will start with 
Mr. Baron. In your testimony, you note the important groundwork 
laid for welfare reform when high quality experiments were con- 
ducted in the eighties and nineties to find the best way to help peo- 
ple move from welfare to work. In fact, this research helped shape 
the successful 1996 reforms which created TANE. 

Beyond TANE, however, the Subcommittee oversees other pro- 
grams that haven’t benefited from the type of experimentation and 
high quality evaluation that led to welfare reform. Given where we 
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are now, in your opinion, what should we he focused on first? What 
would be our number one priority, and is this just the first thing 
we need to focus on and are there any other priorities that kind 
of fall off of that? I would be interested in hearing that. 

Mr. BARON. Thank you. One of the reasons why there is such 
a large body of strong, in many cases, replicated evidence from ran- 
domized trials in Welfare-to-Work is because the Federal Govern- 
ment had in place for many years, starting at the end of the 
Reagan administration through the first Bush administration, and 
then into the Clinton administration, a waiver evaluation policy, 
meaning the Federal Government said to the States, we will allow 
you to do your own welfare reform demonstrations. We, the Feds, 
will waive provisions of law and regulation to allow you to do those 
reforms if, and here was the quid pro quo, you do a rigorous eval- 
uation, usually a randomized evaluation, to determine whether it 
works or not. That policy, that waiver evaluation policy resulted in 
more than 20 large-scale randomized control trials that contributed 
to the important body of knowledge, the evidence that helped in- 
form the work-focused 1999 Welfare Reform Act. 

That kind of waiver-evaluation approach could be used in many 
other programs. The same general concept, it would have to be 
adapted, it could be used in unemployment insurance, in foster 
care, in SSI, and disability insurance, and other areas to allow 
State and local innovation, open the door fairly wide, coupled with 
a requirement for rigorous evaluation to determine which of those 
innovations really work and which do not. 

That is something that — and also importantly, as in welfare, 
with a requirement for cost neutrality, so that you are testing inno- 
vations that are designed to improve people’s lives while not adding 
to the deficit or that are cost saving while also improving people’s 
lives or not causing any harm. 

Chairman REICHERT. Appreciate that. Thank you. 

Ms. Cox, you describe how Utah has a specific division in the De- 
partment of Work force Services focused on building evidence about 
the effectiveness of programs. What did you find most difficult 
about measuring effectiveness? I am going to guess one of the 
things was the disalignment piece that you spoke about. 

Ms. COX. Uh-huh. 

Chairman REICHERT. How did you use this information to 
make decisions about which programs were managed and funded? 

Ms. COX. Well, you know, I have the same question I think you 
raised earlier, do job training programs work or not? I had heard 
a lot of the literature, just like you had, but we wanted to test it 
in Utah. So we did a really rigorous assessment longitudinal, we 
did the whole randomization, and looked at — this is just one of the 
studies, for example, in job training, did it make a difference or 
not. The bigger question was sometimes, and what the type of job 
services that were delivered and when. 

So we found, for example, unpaid internships really didn’t work, 
paid internships didn’t work in all parts of the State except for one 
place, so it was replicating why that worked. We found that occu- 
pational training tied to an employer did work. 

So there are things we found that did and didn’t work, but the 
interesting thing we found, for example, is that when they com- 
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pleted the training was a big variable. We had too many people, 
50 percent of our folks coming into the job training program, start- 
ing it, stopping halfway through or a third of the way through. The 
taxpayers lost the up front cost, and the person doesn’t get the ben- 
efit, so it really forced us to look at new strategies on completion. 
Same thing with TANF participation. What things do people tend 
to participate in without us having to chase paperwork. They tend- 
ed to be employment related and things that related to their lives 
and then which of those ended up helping them become self-suffi- 
cient and moving to jobs. We were able to more narrowly tailor 
what kind of participation activities we focus on. The challenges 
are, there is not a budget, there is not an appropriation in these 
Federal programs. Like I said, 80 percent of our budget at Work 
force Services was Federal. It wasn’t a set-aside amount of money 
saying, here, do evidence-based practices, it is something we had 
to kind of internally create and cobble together some funds for that 
to happen, and there is not an appropriation directly for that. That 
funding piece is a challenge because while we talk at this level, and 
you guys have to make those decisions because you have such a 
huge impact on the Nation, once you get into the operational level, 
people kind of may not take it as seriously, and there is not a re- 
quirement or mandate with some cases like this for that to really 
happen. As we go now, a look at all the other services in State gov- 
ernment will be doing the same thing. 

Chairman REICHERT. Thank you for that answer. My time has 
expired, but I am going to ask Mr. Aos a quick question because 
I know he has to leave and catch an airplane here in a few min- 
utes. Could what you do in Washington State be replicated at the 
Eederal level, and what are some of the key challenges we might 
have to overcome here in the Capital to do so? 

Mr. AOS. Mr. Chairman, I have worked in the State capital of 
Washington for 36 years, so I know that place pretty well. I don’t 
know this place very well, so I am not going to be the one to give 
you advice on can what be done in Washington State be transferred 
to Washington, DC. I think the principle is that it works so well 
in Washington State is that the request for this information is bi- 
partisan. We rarely get a demand for a study that only comes from 
one party or the other. It is almost unanimous votes, that they 
want to find out what reduces crime or what gets more high school 
graduates. 

The other thing that we have done then is that the rigor with 
which we as the people that draw the information go through to 
assess the evidence fairly and to use return on investment analysis 
to rank options because you can find things that work that cost an 
awful lot more money than the benefits they derive, so it is that 
aspect. 

And, then, finally, I would just add that we use that evidence to 
cut programs in addition to add programs that work, and I think 
that that message that the legislature is using evidence to change 
budgeting up and down has resonated around and causes actions 
and responsiveness to the notion so that evidence-based doesn’t 
mean just a code word for spend more money. 

Chairman REICHERT. Thank you for your answer. Mr. Doggett, 
you are recognized. 
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Mr. DOGGETT. Thank you, Mr. Chairman. Mr. Aos, I notice in 
your evaluation it looks like as far as child welfare is concerned 
that the Nurse Family Partnership, the visiting nurse program for 
low-income families, is way out on top as being the most cost-effec- 
tive program. 

Mr. AOS. Yes, it is right near the top of our list. I think it is 
also a buy recommendation from Jon Baron’s group as well. 

Mr. DOGGETT. Exactly. Thank you. I wanted to ask him also, 
what is it about this program that seems to have the most benefit? 

Mr. BARON. The program is very well designed in the following 
sense: It is for women who are poor, pregnant with their first child, 
and most of them are single. They are visited by a nurse, many of 
them — ^you know, they are pregnant with their first child, they are 
concerned about their health, so they are particularly receptive to 
advice from a nurse. The nurse teaches them basic parenting, nu- 
trition, not to smoke or drink during pregnancy. If they are inter- 
ested in practicing birth control, how to do it effectively. One of the 
reasons this program is on the top of Steve’s list and the top of our 
top tier panel’s list is that it has been evaluated in three different 
randomized control trials, in different cities, different ethnic 
groups, actually different decades. 

In all three cases, it was found to produce large improvements 
in life outcomes including, for example, a 40 to 50 percent reduc- 
tion in incidents of child abuse and neglect and hospitalization. 

Mr. DOGGETT. You are aware that the Federal funding for that 
program expires in little more than a year, the Federal Home Vis- 
iting Program. Do you favor its extension? 

Mr. BARON. Definitely. In many Federal social programs, evi- 
dence plays little role in how funds get allocated, whether they are 
formula programs or even most competitive grant programs. The 
evidence-based home visiting program is an important exception to 
that. Evidence, especially for the largest grants plays a central role 
in determining what gets funded. 

Mr. DOGGETT. Thank you very much. 

Ms. Smith, let me talk to you about another innovative program 
that you focused on with Capital IDEA and Project QUEST. This 
is not only about just securing any job that is there, but as you 
mentioned a career pathway so that a person has hopes of not only 
getting a job but getting a job that will help them support their 
family at a livable wage. As I understand the program, again, it 
is not just about how you have become a radiology technician or 
someone who works in semiconductors, but it is about getting some 
counseling to go along with that training to be sure that you are 
able to fulfill all the responsibilities. Can you elaborate a little on 
how those programs work? 

Ms. SMITH. Yes. The Capital IDEA model provides the occupa- 
tional training as a connection with the associate degree program 
or community college program that builds that occupational skill, 
but they also work on building the soft skills that are important 
in the workplace and make someone a successful employee, and so 
through weekly sessions with a career coach, participants go 
through and talk about issues like time management, communica- 
tion and interpersonal relationship skills, and work on building 
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kind of some self-confidence that they can take into the workplace 
and make sure they are going to he a valuable employee. 

Mr. DOGGETT. And how might those programs interface with 
TANF? Is there the potential to assist more TANF recipients, but 
to help them achieve some of the same success that Capital IDEA 
and Project QUEST are already achieving? 

Ms. SMITH. Certainly. Actually both of those programs, as well 
as the Career Advance program in Tulsa, serve TANF recipients al- 
ready. They are part of that low income and disadvantaged group 
that these programs are explicitly trying to move forward in the 
workplace. 

Mr. DOGGETT. Are there other recommendations that you have 
that we should consider as we are renewing and reauthorizing the 
TANF program to assure that more economically disadvantaged 
people actually move into living wage jobs? 

Ms. SMITH. Yes, I think reconsidering the work requirements to 
allow individuals to engage in that longer term intensive skill de- 
velopment that has been shown to lead to higher paying careers 
that actually move people out of poverty would be an important 
change to consider. The emphasis on work first with a very short 
term emphasis on job achieving skills hasn’t been shown to be ef- 
fective in the same way that building an occupational credential 
that employers value has. 

Mr. DOGGETT. And, Ms. Cox, isn’t that also the finding of an 
analysis in Utah that was made last year, that work first is not 
necessarily as important as the training activities? 

Ms. COX. Actually, I don’t think it is either/or. I think it is short- 
term occupational training that is connected to an employer, so I 
think it is bridging both of those worlds. We know that connection 
to the labor force over time is an important indicator based on 
what we saw, and I can’t speak for other States, but for long- 
term — after 4 years there were retained earnings and increased 
earnings. 

So there is a balance. For men, for example, their struggles seem 
to be a little bit different. There is not a lot of men in TANF, but 
our population has gone from 6 percent to 13 percent. Men in 
TANF often have criminal background issues. Sometimes they need 
to get attached to the labor market quickly so they can reengage. 
Most of our TANF recipients get off — 70 percent are off between 2 
and 9 months — so this long-term thing isn’t as critical as maybe for 
the folks in the 30 percent who don’t have a high school diploma. 

So I am always really cautious of this one-size-fits-all and the 
need to really let States give us the policy objectives, the goal of 
the outcome. Do the evidence-based practices at the national level, 
but States need to customize it for their unique demographics. 

Mr. DOGGETT. Thank you all. 

Chairman REICHERT. Thank you, Mr. Doggett. Mr. Renacci, 
you are recognized. 

Mr. RENACCI. Thank you, Mr. Chairman. I want to thank the 
witnesses for testifying today. In my home State of Ohio, an esti- 
mated 1.8 million Ohioans are living below the poverty line. Pov- 
erty in my home State of Ohio has increased by approximately 58 
percent over the last decade despite a stagnant population and a 
whole host of Federal programs created to end the cycle of poverty. 
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So I am glad that we are here today to discuss policies that work 
because, frankly, the people of Ohio and the Nation cannot con- 
tinue on this path. We must find ways to address our struggling 
economy, improve our education system and work force training 
programs, and connect individuals to temporary resources they 
need to succeed. 

With that in mind, and I would like to start with you first, Ms. 
Cox, what should the Federal Government’s role be in social wel- 
fare? And to add on to that, should the Federal Government 
incentivize States to focus on outcome-based programs and should 
funding to States be tied to performance? I know you talked a lot 
about your State and maybe funding toward the State. 

Ms. COX. Well, again, you know, our goal is, even within the 
State of Utah, 25 percent improvement over the next 3 V 2 years, 
and some State agencies are saying that is impossible, but when 
we get into the guts of the systems we are seeing there is a lot of 
capacity there. Having said that, I am all for outcomes and results. 
It is taxpayer dollars, and we need to be accountable. My pref- 
erence is that we are held accountable and then given the flexi- 
bility to design the solutions that work. We spend a significant 
amount of time and energy on demonstration projects at the na- 
tional level and pilot projects, and we at the State just need the 
flexibility. We have people at our doorstep today. I don’t have 5 
years to do a demonstration project. You guys can and give me 
what I need. I need the outcomes and the flexibility to get the re- 
sults today for the people standing on my doorstep. 

So we need that, and then we also really need to emphasize with 
States that they, too, need to be held accountable for results, and 
in many cases, they need to pay more attention to evidence-based 
practices and not just do what feels good. 

Mr. RENACCI. So, in general, you believe that States should be 
given the dollars but there should potentially be incentives tied to 
those outcomes, and it should be outcome based? 

Ms. COX. I would be open — the devil is in the details, or God, 
depending on which way you say it, so it would depend on the pro- 
gram and how it was specifically designed, but it is something I 
wouldn’t be scared of 

Mr. RENACCI. Anyone else want to take a stab at that, what the 
Eederal Government’s role should be? Mr. Baron? 

Mr. BARON. Yes, I think one of the challenges here with holding 
States accountable and so on is that at this point we, meaning the 
country really, and researchers and policy officials really don’t have 
a whole lot of strong evidence, replicated evidence, as David re- 
ferred to, about what works. A lot of times. So there is not 

Mr. RENACCI. But if, in Ohio, we have approximately 58 per- 
cent over the last decade has increased, you know, something is not 
working. 

Mr. BARON. Yes. The system is not working, meaning over time, 
and it is true nationally, even since the early seventies, the poverty 
rate across the United States by various measures has not changed 
a whole lot, despite a whole lot of innovation, a whole lot of things 
going on. What is lacking, I would suggest, are interventions like 
the Nurse Eamily Partnership we discussed before, where the 
strong evidence that has been replicated across different sites that 
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they work. So as a first step doing the kind of innovation and cou- 
pled with evaluation designed to grow the number of proven pro- 
grams might be paramount. In the few cases where there are prov- 
en approaches like the Nurse Family Partnership, just try to scale 
those up more widely. 

Mr. RENACCI. Let me change the pace to outcome because I 
know as a small businessowner before I got here, I set programs 
up and then I looked at the outcomes and decided if they weren’t 
working we would change those programs. So we have programs 
that aren’t working. What are some of the consequences of leaving 
in place government programs that do not work? I mean, why are 
we leaving these in here? Are you saying that we don’t have the 
outcomes yet? I mean, there are certain things that aren’t working. 
What are some of the consequences of leaving those programs still 
intact? Mr. Muhlhausen. 

Mr. MUHLHAUSEN. Well, one of the consequences is that we 
waste billions of dollars and we leave people with no hope, and so 
I think we need to — when we look at programs, especially pro- 
grams that are trying to lift people out of poverty, are the com- 
prehensive effect of various programs, are they trapping people in 
poverty? So if someone who is receiving TANF benefits, food 
stamps, and also a housing subsidy, if they get a job and increase 
their earnings so they get a chance to work more hours, will their 
income increase cause them to lose their housing subsidy? In that 
case, that is an incentive not to gain the additional experience, not 
to gain the additional income through your own labors. So in some 
sense, the combined effect of our entire welfare system can create 
a trap for individuals. 

Mr. RENACCI. Thank you, Mr. Chairman, I yield back. 

Chairman REICHERT. Mr. Davis. 

Mr. DAVIS. Thank you very much, Mr. Chairman. I want to 
thank all of the witnesses for coming. 

I know that the Ranking Member has raised a question relative 
to the home visiting program, the nurse visiting in program. And 
I couldn’t help but smile because I recall that when we were work- 
ing on the Affordable Care Act that one of the provisions that I 
supported very strongly, and actually secured a woman to come 
and testify from the Near North Health Corp. which is a commu- 
nity health center in Chicago. And I run into her quite frequently. 
And I always tell her whenever I do that she was very instru- 
mental in helping us to include that program in the Affordable 
Care Act. And so I was very pleased to hear your analysis and the 
impact of it. 

Let me ask, when programs like that, for whatever the reason, 
are not reauthorized, are not refunded, does that take away or de- 
tract from progress that is being made relative to not only moving 
people out of poverty, but also in helping them improve the health 
status of people and communities that they benefit? 

Mr. BARON. Congressman, funding of the Nurse Family Part- 
nership, which incidentally was launched as a pilot program under 
the Bush administration — proposed by the Bush administration, 
and then scaled up by the Obama administration — that funding, 
there is strong replicated evidence that it improves people’s lives. 
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So defunding it presumably would do the opposite. But I would also 
note that the detail is in the specific type of home visiting program. 

The Nurse Family Partnership has been shown effective, but 
there are many other types of home visiting programs that have 
been shown not to work. There was a Federally sponsored evalua- 
tion, the Comprehensive Child Development Program at HHS in 
the nineties, which was a paraprofessional home visiting program. 
There was a large randomized demonstration that found no im- 
pacts. 

So one of the unique things about the program that was enacted 
is that it had a high evidence standard so that specific home vis- 
iting models, like the Nurse Family Partnership, received priority 
for funding. That is unique in Federal social spending. 

Mr. DAVIS. Thank you very much. I really appreciate that kind 
of work, having been engaged not only in health care but also in 
aspects of dealing with poverty and poverty-stricken people and 
communities for a long period of time. I just find that to be incred- 
ible work. 

Dr. Muhlhausen, let me ask you, if I could — I go through the list 
of different kinds of programs. And I just looked at Supporting 
Healthy Marriages. Do you have any revelations on the impact of 
a program like that, or that specific program? 

Mr. MUHLHAUSEN. Well, there are two programs. Building 
Strong Families and Supporting Healthy Marriages. And I think 
the goal is noble. But both of these evaluations show that both pro- 
grams failed to affect the rate of marriage. So that, considered that 
in that sense, the program is a failure. In the case of the Building 
Strong Families case, the program actually had some negative im- 
pacts. But you have to balance that between, if you look at the site- 
by-site analysis, on the local level in some cities, the program had 
consistent negative effects; but in Oklahoma, while it didn’t boost 
marriage, it actually found some positive impacts on the marital re- 
lationships of individuals participating. 

So you have to learn what happened in Oklahoma but systemati- 
cally, when you look at the entire program, there is not much suc- 
cess. 

Mr. DAVIS. And I think that is so unfortunate because I think 
that marriage does play a significant role in the ultimate organiza- 
tion and development of our society. And many of my friends and 
many people that I interact with don’t have much faith in it, I be- 
lieve. Thank you very much. 

Chairman REICHERT. Thank you, Mr. Davis. Mr. Reed. 

Mr. REED. Thank you, Mr. Chairman. Thank you to our wit- 
nesses. 

Ms. Cox, I wanted to start with you and move to a couple of 
other people on the panel. I am very interested in us starting our 
conversation as we go down this path of TANE reform and other 
reforms as to coming to a common understanding as to what the 
definition of “it works” is. And I want to have a clear under- 
standing from the panel as to — especially you, Ms. Cox — out in the 
field, on the frontline, in the States. How is it presently defined to 
be “a success” under these programs? I have heard things such as 
getting people out of poverty. It is as easy as defining people who 
get the benefits, who actually receive a check. So I just want to get 
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clarification from you on that. And then if you have any guidance 
or recommendations as to what is a good working definition of de- 
fining success. 

Ms. COX. It is a really great question, because we talk about it 
in such broad terms, self-sufficiency and moving people out of pov- 
erty. But what do we mean by that? TANF — in Utah, is 50 percent 
of poverty level. Food stamps has a different eligibility criteria. 
Medicaid now we know with the ACA reform is going to take you 
to up 130 percent plus. So at what point does the Federal Govern- 
ment mean “out of poverty” because there are so many different 
contradictory definitions of what that means. In Utah, again, with 
over 90-plus programs for TANF, our definition for quality is 
throughput divided by operating expense so we can get a quality 
cost per case. It is the number of positive closures we have, the 
percentage of those that are placement and employment, right? Be- 
cause sometimes it is so easy for a State to say, oh, they got mar- 
ried, they got Social Security. We want to focus on employment be- 
cause we know in the long term, that 4 or 5, 6 years down the 
road, that gives them a better chance at self-sufficiency divided by 
those costs. We can hit that. That is step one. But if our entire 
caseload of people who are on low-income services — our TANF case- 
load is a drop in the bucket. We spent a lot of time on TANF. But 
in our State, it is 6,000 to 10,000 of cases — that is not individuals — 
and our entire caseload is more like 200,000. 

So the broader question is what is the policy objective for Med- 
icaid, for food stamps, for child care? Because if we don’t drive 
those policy objectives, TANF isn’t going to move the whole system. 
TANF has been a success. We know in the last 16 years, caseloads 
have declined almost by 50 percent. We can continue to improve it. 
We know that. I can get people off of TANF. But moving them off 
food stamps and Medicaid into true self-sufficiency, that is a much 
broader public policy agenda that has yet to be defined. So for us, 
it is benefit. Their case is closed. They are off benefits. And they 
have a job. That is the ideal scenario for us. 

Mr. REED. That is the ideal. Okay. Mr. Muhlhausen. 

Mr. MUHLHAUSEN. I think one thing we need to think about 
is policy significance versus statistical significance. We like to come 
here, and especially me, we like to talk about statistical signifi- 
cance, a particular program, boost the wages up let’s say a head 
of the household by $1,000 every year. This finding was statis- 
tically significant. Well, what is the policy significance of that? 
That additional $1,000, does it necessarily raise that family above, 
say, the poverty level? 

So sometimes these programs we are talking about are actu- 
ally — while they do have a positive impact, and I can say statis- 
tically significant, meaning we believe the results actually occurred 
and were reliable. But sometimes the size of the effect is actually 
not that meaningful as far as changing the individual’s life. So I 
think we need to think about not only statistical significance, but 
also policy significance. Is a program, let’s say, moving somebody 
above the poverty threshold? And are they on a trajectory where 
they are not going to be dependent on receiving future government 
services? 

Mr. REED. Go ahead, Ms. Smith. Please. 
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Ms. SMITH. I think a standard way of looking at whether you 
are making a difference for families is, are they in stable employ- 
ment with rising earnings? That is what lifts families out of pov- 
erty. And it has been shown to have a really positive impact on the 
children of those families as well. They do better when they have 
that sort of financial security. 

Mr. REED. I am getting — ^because I am running out of time and 
I don’t mean to cut you off. 

I am getting consensus from the panel that having a simple defi- 
nition of, we are going to have x number of dollars or x number 
of benefits in the hands of a recipient, is probably not the best defi- 
nitional program or definition of success. Am I misinterpreting any- 
thing anyone is saying there? With all the nods of the head, it 
sounds like there is agreement there. So I appreciate that because 
a lot of times, I have conversations with Members up here and they 
are just as simple as. Well, if we get x number of dollars in the 
hands of a recipient, that is a win. That is a success. And clearly 
it is much broader than that. My intention in doing the work here 
is to improve lives, not just give benefits to people. So I appreciate 
that. With that, I yield back. 

Chairman REICHERT. Thank you. Mr. Kelly. 

Mr. KELLY. Thank you, Mr. Chairman. Again, thank you all for 
being here. 

One of the things I am trying to understand — and it is not for 
a lack of investment, is it, on the part of the government? I am 
looking at the investment that we make each year. And if the num- 
bers I am hearing are right, it is about $600 billion a year that 
goes into trying to lift people out of poverty or support the most 
vulnerable in our society. That is a lot of money. But I think our 
concern is, what is the return on that investment? What are invest- 
ing these dollars in? And the idea was to lift people out of whatever 
conditions they were in. 

So, Ms. Cox, I heard you say that part of the problem is, some- 
times there is a negative incentive because once you get out of one 
level, then you go into another. And all of a sudden, it is like, well, 
this doesn’t work in my best interest. 

So best practices, are you able to share those with each other? 
I know there are a lot of programs. Your State is doing some things 
that maybe other States should do. 

Do you have the ability to communicate back and forth? 

Ms. COX. Yes. There are associations and forums. But you know, 
when you are in the trenches — and especially during the recession, 
our caseloads increased by 63 percent. So we were just treading 
water to get through that and get people back to work and contain 
costs. We were able actually to reduce our costs by 33 percent 
while our caseloads increased by 63 percent and improve our time- 
liness. But it is really difficult. It is nice to have the luxury, with 
all due respect, to analyze this stuff for 4 or 5 years. But that isn’t 
a reality when you are on the ground. 

So we need people — these brilliant people here to inform us, to 
educate us. We are committed to evidence-based practices in Utah. 
It is the only way we will hit our 25 percent improvement in some 
of our agencies. But sometimes we need States just to be able to 
innovate to get to the clear results. Because if we don’t have time 
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to do the evidence-based practice, and we are not allowed to inno- 
vate, we are stagnant, and we can never make progress. So that 
is the bind States are in. We need evidence-based practices. But 
when it is not there, we need the flexibility to innovate to get the 
results for you guys and for the taxpayers. 

Mr. KELLY. And you mentioned innovation. My friend David 
Bradley has talked to me many times. He is with the National 
Community Action Foundation. And he talks about the ability to 
look at innovation, to look at the performance data, and then also 
local control of these dollars. And again, I have a friend in Sharon, 
Pennsylvania, by the name of Ron Errett who runs a program up 
there. It is the Community Action Partnership of Mercer County. 
So I have seen locally in the district that I represent a lot of pro- 
grams that work really well. And I think we have got to be careful 
that we don’t paint everybody with the same brush and say, we are 
wasting this money. Nothing is going the right way. I don’t think 
that is true. And we referenced the Nurse Family Partnership, how 
much that has worked. 

But it does come down to, how do we get that? How do we get 
the innovation message out? How do we share those practices? Mr. 
Baron, you made some comments about that when you talked 
about the ability. And the results speak for themselves. When you 
see something good, how do you get that out? Because I have got 
to tell you, in my district, I was able to look up and down north- 
western Pennsylvania. The poverty level is probably somewhere be- 
tween 25 and 30 percent. Not really mattering what town you go 
into, big cities, little towns, it is about the same. This poverty thing 
is something that is really troubling that we spend all this money, 
but we haven’t gotten any results and we don’t see that happen. 
I know part of it is the economy not bouncing back. And maybe we 
are spending a lot of time criticizing programs and not coming up 
with leadership programs or strategies that lifts everybody. 

Ms. COX. Can I make one more point on that just to be kind of 
bossy? 

Mr. KELLY. Sure. 

Ms. COX. There are associations, administrators, that are always 
connecting and going to conferences and talking about best prac- 
tices. But part of the challenge is, we have the evidence-based prac- 
tice. But if you were to look and do our mapping and look at an 
employee’s time. They have 8 hours a day. And let’s say we even 
know what the evidence-based practice is. We know that they 
should be doing X, Y, Z every day with their customers to get the 
impact. If you were to map out how much time they actually spend 
doing that, in some cases, you will find 10, 20, 30, 40 percent of 
their time is actually spent on the evidence-based practice. The rest 
of the time it is spent on compliance, recording, paperwork, a lot 
of other stuff. So can you imagine the capacity to impact low-in- 
come individuals? If we could just double the time on the ground 
in your operations and in your systems design of what people 
do 

Mr. KELLY. I am going to agree with you because I have got to 
tell you. I run a private business. We do the same thing with our 
business trying to be in compliance. If I could just do what we are 
designed to do every day and not worry about being in compliance 
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with the Feds, the local, and the State, we would probably get a 
lot more done. Mr. Baron you were going to say something? 

Mr. BARON. Yes. One of the challenges in sort of identifying and 
sharing best practices, things that really work is that a lot of pro- 
grams, almost every program claims to be evidence-based and 
backed by strong evidence and effective. And the truth is, while 
some of them are, when most programs — even those that are 
backed by pretty good evidence — are subjected to a definitive eval- 
uation, many of the promising findings are not reproduced. Some- 
times they are. So you do have some examples of effectiveness, but 
many times they are not. 

Steve Aos’ organization, the Washington State Institute, does a 
valuable service by trying to distill what is really backed by strong 
evidence from others that are not. But there are some instances. 
There was a program that the reemployment and eligibility assess- 
ment program at the Department of Labor which has been shown 
very effective in a four-State randomized control trial with large ef- 
fects on employment outcomes and reductions in spending. 

Mr. KELLY. Thank you. 

Chairman REICHERT. Thank you. Mr. Crowley. 

Mr. CROWLEY. Thank you, Mr. Chairman. Thank you for yield- 
ing me the time. 

I think we are all good stewards. I think we all want to be good 
stewards of our taxpayer dollars and certainly we want to put 
money where we think it best works for the American people. 

Mr. Baron, I just want to follow up my comments of Mr. Doggett 
as well as Mr. Davis as it pertains to the Nurse Eamily Partner- 
ship is the kind of program that has been proven to get results. As 
you know, I know — I am not so sure my colleagues know — that the 
program started in my home State of New York — in fact, in Elmira, 
in my good friend, Mr. Reed’s district. And I have long followed the 
very impressive work that they have been involved in replicating 
this model and achieving very significant positive outcomes. Reduc- 
tions in child abuse and neglect, better educational outcomes for 
children, and greater likelihood of economic stability for the moth- 
er, these are just some of the results that actually save the govern- 
ment money in the long term. 

Mr. Baron, do you think that is correct, that it has an effect in 
terms of saving taxpayer dollars in the long run? 

Mr. BARON. In this case, I think the answer is yes, especially 
with the Nurse Eamily Partnership. One of the trials that was 
done in Memphis, Tennessee, measured not only the impacts you 
are describing, like child hospitalizations and educational outcomes 
for the children, it also measured participants’ use of government 
assistance — Medicaid, food stamps, welfare over a 12-year period. 
And this was not a projection. They measured use of government 
assistance in the treatment versus control group. So this was a 
credible finding. And it found that the program produced savings 
that more than offset the cost of the program. 

Mr. CROWLEY. Mr. Baron, would you also agree that programs 
like this particular one are an example of the importance of looking 
at successes not just in the short term, but in the greater awards 
to our society down the road? And not just immediate. 
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Mr. BARON. Yes. Because some of the effects are longer term. 
Some of the effects were short term. There were immediate effects 
on reductions in child maltreatment and hospitalizations. But there 
were also longer term. 

Mr. CROWLEY. So leading to a much longer and productive life 
for the child in the long term. 

Mr. BARON. That is correct. But a slight nuance on that is that 
some programs — especially in work force development — they 
produce short-term effects, which dissipate over time. And so for 
those kinds of programs, it is important not only to measure the 
short-term effects which are sometimes large, but whether it pro- 
duces sustained effects on the amount of 

Mr. CROWLEY. I appreciate your comments. 

Better access to health care coverage, nutrition programs in early 
childhood, we believe leads to greater health and reduced medical 
costs later in life. And that is what we are talking about at this 
point in time as it pertains to the Nurse Eamily Partnership. 

Strong education, mentoring, and family supportive programs re- 
duce incidents of criminal activity and school dropout rates as well. 
I think too often, we have a tendency and I think is a willingness 
to cut a program for ideological purposes not because it is what is 
in the best interest of our country. We see it in the Affordable Care 
Act. We have seen it in the farm bill nutrition programs. We have 
seen it as well in the social services block grants and others. It 
seems like the lesson here today is that we need to carefully invest 
in the programs that work and really put a lot of thought and 
study into our budgeting process. 

So I would think the budget approach we have recently seen with 
policies, like sequestration, is the exact opposite of what we ought 
to be doing. Blunt across-the-board cuts and not replacing them 
with a thoughtful plan that grows the economy certainly doesn’t fit 
with the evidence-based approach that seems to be the rec- 
ommendation of the witnesses here today. I would hope that my 
colleagues not just on this Committee but on the Budget and Ap- 
propriations Committees as well and every other Committee draw 
lessons from this hearing. And I look forward to more a construc- 
tive conversation and hearings like this in the future. 

With that, I yield back the balance of my time. 

Chairman REICHERT. I can assure my good friend that I am 
very interested in evidence-based results as an old time cop. So 
with 33 years in that field, I am looking for evidence. Mr. Young. 

Mr. YOUNG. I want to thank the chairman for holding this hear- 
ing on what really works. I think it is incredibly important. It may 
be mundane to some people. It may be boring to others. Metrics 
and data and all these other things. But let me begin by defining 
the challenge and perhaps identifying the opportunity or opportuni- 
ties as I see them here. My interest in this topic actually emerged 
the second I found out I was going to get on this committee. I sus- 
pected it wasn’t unlikely I was going to be on the Human Re- 
sources Subcommittee, and that is a good thing. We deal on this 
committee with what some regard as unsolvable social pathologies. 
And I sort of refused to believe that. 

So I know a number of pilot programs over the years have taken 
place across our 50 States. So I directed Members, associates on 
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our team to try and identify the results of those different pilot pro- 
grams and find some central repository where we could find out 
what really works and under what circumstances. It was incredibly 
tedious work; and frankly, there was no such repository. There was 
not a navigable Web site. There wasn’t a particular organization 
that seemed to have answers about what really works and what 
data you could look at and what works under different cir- 
cumstances. 

I found this frustrating as a policymaker. And I think at the 
State level, local level, not for profits, academics, think tankers, 
and so on, could all benefit from more clarity here and a more ro- 
bust collection of data. 

What are we left with without this sort of repository of accessible 
data? Well, we make policy based on ideology, on politics, on anal- 
ogy, sometimes anecdotes, the news of the day; but we don’t really 
make decisions based on the hard data. So I started doing a bit of 
reading. I discovered that in 1988, under the AFDC Reform Act of 
that year — not often talked about — there was a requirement that 
data be collected on the recipient population of AFDC. That data 
years later — oftentimes it does take years for this information to be 
teased out — that data established the intellectual groundwork for 
a bipartisan reform of the AFDC program, now the TANF program. 

We need to make similar efforts in other areas. We need to do 
more evidence-based policymaking. 

Mr. Baron, thank you. It is so great to see you here today. You 
are really a gift to this conversation. I thank everyone else as well. 
And we have convened a group of people to discuss this topic. It 
is my hope that we can come up with a more systematic way of col- 
lecting data in a number of different areas and promulgate and dis- 
seminate that data to others for the purpose of research and also 
for the purpose of evaluation so that when innovation occurs at the 
State level, we will know if, in fact, it is working and then share. 
It is an iterative process. Share what is learned with others. And 
that will enable us to do very creative things, like social impact 
funds, pay for performance, pay for success in some of these social 
areas, the same sort of thing we do in, say, the transportation sec- 
tor, performance-based contracting. 

Mr. Baron, you said Congress could take steps in your statement 
toward what I have envisioned, I believe, by authorizing and en- 
couraging agencies to allow greater research access to administra- 
tive data with appropriate privacy protections so as to facilitate 
low-cost rigorous evaluation. 

I have two questions for you. First, what data is the Federal Gov- 
ernment failing to collect that we ought to be collecting about bene- 
ficiaries of government programs? And second — and I see my time 
is running down, so you can submit this in writing. But second, I 
did want to get it on the record, what data is already being col- 
lected by the Federal Government, such as receipt of government 
assistance, employment status, earning status that we should re- 
lease to the public for research purposes so it is not just our bu- 
reaucrats who are armed with all the information. 

Mr. BARON. I would like to submit a response in writing. But 
also a very quick answer to the second part of your question. This 
Subcommittee took a major step forward, we believe. We were very 
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supportive of the subcommittee’s action to increase researcher ac- 
cess with appropriate privacy protections to the National Directory 
of New Hires, the NDNH data, which has, at the Federal level, the 
employment and earnings records. You can use it to measure em- 
ployment and earning records — earning outcomes for participants 
in any study. It should be more widely available. It would reduce 
the cost of some of these rigorous studies by a factor of 10 or more. 

Mr. YOUNG. Thank you. 

Mr. REICHERT. Thank you, Mr. Young. Thank you to all of the 
witnesses for being here today and taking the time to be with us. 
And we have finished this almost in record time, just in time for 
votes. I thank the Members for being here too. We look forward to 
working with you and reaching out to you and asking you ques- 
tions, more questions that will come, I am sure, as we struggle 
with trying to find solutions here that are evidence-based, where 
we hold programs accountable, make sure that we are really help- 
ing those people who need the help and ensure that they are mov- 
ing up that economic ladder as we all hope that they do. 

So if Members have additional questions for the witnesses, they 
will submit them to you in writing. And we would appreciate re- 
ceiving your responses for the record within 2 weeks. The Com- 
mittee stands adjourned. Thank you. 

[Whereupon, at 5:30 p.m., the subcommittee was adjourned.] 
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Dear Chairman Reichert and Ranking Member Doggett: 

Family Equality Council is the national organization that supports and represents the three 
million parents who are lesbian, gay, bisexual, and transgender (LGBT) and their six million 
children across the United States. We embody a community that works hard and strives in 
every way to create a safe, stable, and loving environment for our families. When children are 
living in poverty, the root causes need to be addressed. Our organization works in many areas 
to help decrease poverty among our community's families, including advocacy in foster care, 
adoption, employment discrimination, health care, immigration, and public benefits. We are 
pleased to have the opportunity to submit testimony for this .subcommittee hearing on efforts 
to help families support their children and escape poverty. 

In his opening statement at the June 18, 2013 hearing Reviewing How Today's Fragmented 
Welfare System Fails fo Lift Up Poor Families, to which we also submitted testimony, Chairman 
Reichert shared his belief that "lojur goal is to help more low-income families leave poverty and 
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achieve the American Dream, That's not a Republican goal, ora Democrat goal. And the fact 
that too many of our fellow citizens have seen that goal slip from their grasp in recent years is 
our cal! to action." This is a noble goal and one that LGBT families share, in order to effect this 
rise from poverty, some individuals need the assistance of social safety net programs provided 
fay our federal government. Unfortunately for LGBT families, the application and acceptance 
process is cumbersome and often unsuccessful. 

As a community, LGBT families are economically vulnerable as a result of many societal factors. 
According to a recent Williams institute report, LGBT individuals, families, and their children 
endure higher levels of poverty than their heterosexual counterparts.^ This report incorporated 
data from four separate studies comparing poverty rates between heterosexual and LGBT 
people Individually and as couples.^ Children in same-sex couple households are almost twice as 
likely to be poor. Children in male same-sex couple households have a 25.4% poverty rate and 
19.2% of children in female same-sex couple households live in poverty. This is compared to the 
12.1% of children in married, different-sex households, Children in a same-sex couple family are 
also more likely to live with incomes below 200% of the Federal Poverty Line {FPL).^ 

There are a variety of reasons why LGBT people are more vulnerable to poverty. The 
community tacks legal protections from employment discrimination. LGBT families are excluded 
from tax benefits and many government financial and health care assistance programs. LGBT 
people are working to provide for themselves and their families but may need some financial 
assistance and tax credits to make ends meet. Data in the Williams Institute report indicates 
that LGBT people are receiving government aid in slightly higher percentages as a result of 
these hurdles that they face to sufficiently support themselves.** That is not to say that being 
approved for welfare services is not alone a challenge, especially for same-sex couples with 
children. The focus of this testimony wit! be on the challenges many LGBT families face when 
attempting to access welfare safety net programs and worker tax benefits. 

Qualification for government benefits tends to be tied to marital status. The recent Supreme 
Court decision invalidating the OOMA definition of marriage should make qualification for 
federal benefits a bit easier for same-sex couples legally married in their home state. This 
change, however, does not help same-sex couples and their children in the 37 states in which 


’ M.V. Lee Badgett, Laura E. Durso, & Aiyssa Schneebaum, “New Patterns of Poverty in the Lesbian, Gay, and 
Bisexual Community," The Williams Institute, UCLA School of Law, June 2013, available at 
v.'iTpi/'i^i!ij.3.’T-?LG>Titut;eJa_w._u_c!3.ed_u./vyp-coj}t_ent/_uo!p3dsVLGB;_Po 

^The report includes data derived from 2010 American Community Survey, 2006-2010 National Survey of Famity 
Growth, 2007-2009 California Health Interview Survey and a 2012 Gallup Daily Tracking Poll. 

’ Supra note 1, 

Ud 
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same-sex marriage Is still inaccessible. This discrepancy in treatment between the children of 
same-same and opposite-sex parents creates a two-tiered system of social safety net programs. 
A simple application and qualification process exists for different-sex couples and a time 
intensive, embarrassing, and often fruitless process exists for LGBT families.^ 

At the previous hearing in this series. Congressman Doggett introduced the topic of the Earned 
income Tax Credit (EITC) and stressed its importance as a tool to support working families. 
Mention was repeatedly voiced by congressmembers and witnesses of this tax benefit and its 
ability to lift hardworking Americans out of poverty. For the LGBT community, however, this tax 
credit is difficult to obtain. This credit is intended to assist low wage earners with children. The 
definition used to determine the relationship of children, however, denied eligibility to many 
same-sex partners raising children. For the EITC purposes, the child must be the tax filer's son, 
daughter, stepchild, eligible foster child, adopted child or a descendant of any of them or 
his/her brother, sister, half-brother, half-sister, stepbrother, stepsister, or a descendant of any 
of them.^ These requirements do not take into consideration families living in states where 
joint, second parent, or step-parent adoption is rare or completely unobtainable for LGBT 
parents. 

Children fall through the cracks when government programs refuse to recognize their families 
as families. These programs, providing monetary assistance, food, health insurance, childcare 
assistance, and tax credits use narrow definitions of family to determine eligibility.^ Actual 
household size may not be taken into account when assessing the means of support available 
to the family, in addition, the lack of universal adoption and parentage laws creates confusion 
In the application for benefits and ultimately bars access to social service programs to children 
of same-sex couples. 

Take, for example, a household consisting of two parents and two children, Post-DOM A, if this 
couple is married in a state where same-sex marriage is legal, the federal government will 
recognize the relationship for the purpose of social services. This qualification may not 
automatically apply, however, to LGBT couples in non-same-sex marriage states, or states 
without comprehensive access to parent-child relationships. While this family would appear to 


' Burns, Crosby, Center for American Progress, The Federal Budget and Gay and Transgender Families: Not All 
Families Are Equal Under the Law April 2012, found at http;//www.ameriC3nprogress.org/wp- 
content/'uplQads/issues/2012/04/pdf/lgbt__budget.pdf. 

® http://www.ir5.gov/tndtviduals/EiTC,-£arned-fncome-Tax-Credft,-Que5tions-and-Answer5 

’ Movement Advancement Project, Family Equality Council and Center for American Progress, "All Children Matter; 
How Legal and Social Inequities Hurt LOST Families {Full Report)," October 2011, p. 52, condensed version available 
at www.lgbtmsp.org,r!gbt'families. 
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be a four-person household, the household size is reduced to three for the calculation of 
assistance needed, lowering the amount of vita! aid for which the family would qualify. 

Similarly, this family could be denied the EiTC. if the partner who is working to support the 
family is not the biological, foster, or adoptive parent of their children, he or she would not 
qualify for the credit to assist the whole family. This credit meant to bring hardworking 
Individuals out of poverty would be refused to the LGBT applicant as a result of the narrow 
relationship requirement. 

These financial assistance challenges create a vicious cycle for LGBT families. Both parents may 
want to work full time but cannot because they do not earn enough to afford the child care 
costs associated with being out of the home and, because of a lack of legal relationship with 
their child, do not qualify for childcare assistance. Suppose a same-sex couple used 
reproductive assistance to conceive their child and then arrived at a point in their lives when 
they needed to apply for TAN F assistance. TANF procedures call for location of a second 
biological parent in order to determine benefits. Penalties can be applied to these parents for 
not being able to identify the second biological parent as well as delays in processing while 
helping a caseworker understand their situation. Perhaps only the non-iegal parent has 
healthcare through his/her employer and therefore cannot cover the children but the family 
does not qualify for CHIP because of the aforementioned means test. The family must pay for 
private insurance or pay exorbitant medical bills for their uninsured children. Either way, their 
modest income is now stretched even farther. 

Some strides are being made to combat the challenges faced by LGBT people in need of 
assistance. Food assistance programs use a broader definition of household size. For the 
purposes of free and reduced lunches, WIC, and SNAP, there is no requirement that applicants 
be legally or biologically related to be included in the household.^ The benefits go to those who 
sit at the table and eat together: the definition of family, Also, the Free Application for Federal 
Student Aid (FAFSA) is being revised to require applicants to list both of their parents if they live 
together, regardless of gender, marital status, or sexual orientation. This change will make 
applying for financial aid less complicated for the applicant and the processors. Clarifications to 
the Family Medical Leave Act have made it clear that a biological relationship is not required to 
qualify for leave to care for an individual when acting in loco parentis.® 


' Id. 

'Lepnick, Nancy J. Deputy Administrator, Administrator’s interpretation 2010-3, U.S. Department of Labor, Wage 
and Hour Division, http://'vVww.dol.gov/WHn/op!nion/adm!nintrprrr!/FMLA/2Qi0/FMLAA!2t310_3 .prif. 
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Changes to the narrow definitions of family for the purpose of applying for welfare assistance 
and tax credits will help break the vicious cycle. Timely processing of applications and accurate 
assessment of need could shorten the time a family needs to receive these forms of assistance. 

The solution to many of these shortcomings Is to create consistent, broad definitions of family 
for ai! federal programs. For TANF, the definition would expand the "assistance unit." For CHIP, 
the definition would include same-sex partners and children for whom an adult is standing in 
loco parentis. For Medicaid and SSI, the definition would include de facto parents.^® 
Relationships, for the EiTC, would include children of same-sex partners residing together as a 
household. 

The Employment Non-Discrimination Act (ENDA) is currently undergoing consideration in the 
Senate (5. 815] to eliminate some of the obstacles LGBT individuals face in the workforce which 
should lead to higher LGBT employment rates and less need for social safety net benefits. 
Passing this crucial legislation would go incredibly far towards stabilizing economic security for 
LGBT parents and their children. The bi-partisan sponsorship of the Every Child Deserves a 
Family Act {ECDF} (H.R. 2028) is a step toward allowing LGBT couples to create families 
nationwide and place our nation's children in secure, financially sound, loving homes. Family 
Equality is working on these projects and would be happy to share our work and family stories 
with you- 

The American family comes in various forms. The one thing we have in common is a desire to 
provide a secure and stable home for our children. No matter how hard we work to accomplish 
this goal, times can get rough. The welfare system exists to assist families during these rough 
times. Together we can ensure the best outcomes for these children and families, while making 
sure that government aid is distributed as efficiently as possible. 

Thank you again for the opportunity to testify on this important issue. If you have any questions 
about our testimony, please email me at ehecht@familyequaiity.org or contact me by phone at 
202-496-1285. 

Thank you, 


Emily Hecht-McGowan 


Supra note 7. 
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DATE: July 26. 2013 

TO: Chair Reichert. Subcommittee on Human Resources of the House Ways and Means Committee 

Sl^B JECT : Testimony submission for the Hearing on Evaluating Efforts to Help Families Support their Children 
and Escape Poverty, July 17. 2013 

Hie American Evaluation Association ( AEA) is pleased to write hi support of the work of this committee regarding 
Evaluating Efforts to Help Families Support their Children and Escape Poverty. This is au important area that has a 
rich tradition of applied research and comprehensive evaluations of experiments conducted in partnership with state 
welfare directors to meet information needs. These studies have provided mvaluable information to those who make 
decisions about policies and programs ooucermng families and poverty. We applaud your inteiest in evaluating 
efforts to alleviate j}o\'eity and wish you the very best as you go forwaivi. 

As the priuiaiy' evaluation a.ssociation in the U.S., we are writing to offer our assistance to the cominiftee on the 
issues you are reviewing. .^EA has approximately 8.000 members representing all 50 states and the Di.strict of 
Columbia as well as over 60 foreign couuti'ies. I have attached exceiysts from An Evaluation Roadmap for a 
More Effective Government, a document we developed to help governments engage m effective evaluation. I 
think you will find it very useful. 

AEA encourages Congress to eusiu'e more social progmins are evaluated to determine then impact and to consider 
how liigli-quality evidence can best be used to inform the design of social programs at the fedeird level. In 
pai'ticulai', AEA believes that Congress should ensiue that sufficient resources are made available for quality 
evaluation and encomage agencies to set guidelines for the conduct of evaiuahons, including the development of 
evaluation agendas and plans for dis.seininating evaluation fuKlmgs to staff and the public. 

AEA has long sponsored academic research and fostered exchange.s among evaluation practitioneix regarding 
evaluation methodologies that can most effectively shed light on the causes and remedies for poverty^ If we can 
be of as.sistance, or if you need more information on our organization, please do not hesitate to contact me at 
(jody.fitzi>atiick@ucdenver.edu) or Dr. Cheiyl Oros. otir senior advisor for evaluation policy 
(evaluafioiipolicy@evai.org; 540-894-4014J. 


Sincerely, 



Jody Fitzpatrick, Ph.D. 

President, American Evaluation .Association 
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An Evaluation Roadmap for a More Effective Government 

September 2010 

The American Evaluation Association (AEA) is an international professional association of evaluators 
devoted to the application and exploration of program evaluation, personnel evaluation, technology, and 
many other forms of evaluation. Evaluation involves assessing the strengths and weaknesses of programs, 
policies, personnel, products, and organizations to improve their effectiveness. AEA has over 6,500 members 
representing all 50 states In the United States as well as over 75 foreign countries. 

Mission: To improve evaluation practices and methods increase evaluation use, promote evaluation as a 
profession and support the contribution of evaluation to the generation of theory and knowledge about 
effective human action. 

In keeping with our mission, the American Evaluation Association hereby describes its vision of the role of 
evaluation in the federal government. We provide a roadmap for improving government through evaluation, 
outlining steps to strengthen the practice of evaluation throughout the life cycle of programs. 

Evaluation is an essential function of government. It can enhance oversight and accountability of federal 
programs, improve the effectiveness and efficiency of services, assess which programs are working and 
which are not, and provide critical information needed for making difficult decisions about them. 

The Challenge 

Like all governments, the United States government faces challenges in both foreign and domestic policy 
arenas. Today, these challenges span subject areas such as national security, foreign aid, energy, the 
environment, health care, education, and the economy. Program or policy interventions are typically 
developed in response, in an effort to mitigate, resolve, or better understand the problems involved. 

To determine the merit, quality, and usefulness of these interventions, credible information is needed about 
what the program or policy in question has achieved and at what cost. Such information is crucial if 
government officials are to ensure that the chosen interventions are working, that taxpayers' money is being 
spent wisely, and that the government is accountable to the public for the interventions and their results. 

Why Program Evaluation Is Essential 

Evaluation involves assessingthe strengths and weaknesses of programs, policies, and organizations to 
improve their effectiveness. It provides a useful and important tool to address the need for credible 
information, well-grounded decision making, and governmental transparency. Within a government context, 
the legitimacy of evaluation can be seen as deriving from the structure of the government it serves and from 
the functions it fills. 
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In the United States, evaluation can serve information needs that arise witlnn any of the three branches of 
government. For ex.araple, evaluation can provide information about a new program’s initial outcomes, 
allowing for better management within the Executive Branch. It can also be used to assess the relative 
performance of a set of alternative policy options, informing legi.slative deliberations. The courts may cite 
evaluation findings as a basis for their judgments. More fundamentally, evaluation can contribute the 
evidence needed to support the system of checks and balances esteiblished by the United States Constitution. 
For example, evaluation enables congres.siona! oversight and executive accountability, along with the 
development of new knowledge, innovation, and organizational learning in both branches. This commitment 
to accountability and tran.sparency, on the one hand, make.s evaluation essential to democratic government 
and, on the other, requires evaluation in a government context to be independent and to resist advocacy for 
particular po.sitions. 

Evaluation provides needed feedback for managing any program. It uses systematic data coiiecfion and 
analysis to address questions about how well gov'ernment programs and policies are working, whether they 
are achieving their objectives, and, no less importantly, why they are or are not effective. Evaluation 
produces evicience that can be used to compare alternative programs, guide program development and 
decision making, and reveal effective practices. By its very nature, it supplies the publicly accessible 
information that is at the heart of transparency and open government. 

Since the inception of modern program evaluation, federal agencies have conducted many evaluations and 
applied their results to medee reasoned program decisions. But for the most part, these evaluations have been 
sporadic, inconsistently applied, and inadequately supported. The units formed to conduct evaluations too 
often are short lived and under re.sourced. Training and capacity building for evaluation have been 
inconsistent across agencies and, in many cases, insufficient to achieve the needed evaluation capacity and 
sustain it over time. 

Yet there is a .strong ca.se to be made for a commitment to evaluation a,s an integral feature of good 
government, whether the goal is better performance, stronger oversight and accouutabiiiQ', or more data- 
informed and innovative decision nialdng. The lessons learned in agencies that have applied evaluation 
constitute a solid knowledge base upon which to buiicl. 

The U.S. government would benefit significantly from using program evaluation to 

• Address questions about current and emerging problems 

• Inform program and policy planningefforts 

• Monitor program performance 

• Pmvide timely feedback to decision makers to make changes when needed 

• increase accountability and transparency 

• Reduce waste and enhance efficiency 

• Improve programs and policies in a systematic manner 

• Support major decisions about program reform, e.xpansion, or termination 

• Assess whether existing programs are still needed or effective 

• Identify program implementation and outcome failures and successses 

• Identify innovative solutions that work 

• Inform the development of new programs where needed 

• E.xamine the requirements for the transfer of promising programs to new sites 

• Share Information about effective practices across government programs and agencies 
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The key is to make program evaiaation integral to managing government programs at all stages, from 
planning and initial development, through startup, ongoing implementation, appropriations, and 
reaiithorization. In short, what is needed is a transformation of the federal management culture to one that 
incorporates evaluation as an essential management function. 

Recommendations 

We recommend that each federal agency adopt the following framework to guide the development and 
implementation otits evaluation programs. 

Scope and Coverage 

• Conduct evaluations of public programs and policies tiiroughout iheir life cycles and use evaluation to 
both improve programs and assess their effectiveness 

• Evaluate federal programs and policies in a manner that is appropriate for program stewardship and 
useful for decision making 

• Build into each new program and major policy initiative an appropriate framework to guide 
evaluations throughout the life of the program or initiative 

• For existing programs, assess what is already known and develop evaluation plans to support future 
decision making 

Management 

• As-sign senior, experienced evaluation officials and managers to administer evaiitation centers or 
coordmato evaluation functions at appropriately high levels of government agencies 

• Prepare annual and long-term evaluation plans to guide decision making about programs 

• Provide sufficient and stable fimding to support professional evaluation activities 

• Coordinate and communicate about evaluation efforts across agencies with overlapping or 
comp!emenrat7 missions 

• Develop written evaluation policies across and within federal agencies that can guide evaluation 
efforts and help ensure their quality 

• Ensure that evaluation units and staff receive high-level, public, and consistent support 

Quality and Independence 

• Develop and adopt quality standards to guide evaluation functions consistent with the American 
Evaluation Association's Guiding Principles for Evaluators 

• Promote the use and further development of appropriate methods for designing programs and 
policies, monitoring program performance, improving program operations, and assessing program 
effectiveness' and cost 

• Safeguard the independence of evaluation design, conduct, and results 

• Preserve and promote objectivity in examining program operations and impact 

Transparency 

• Consult closely with Congress and non-federal stakeholders in defining program and policy objectives 
and critical operations and definitions of success 

• Disseminate evaluation findings and methods relating to public accountability to policy nialcers, 
program managers, and the public 

• Create clearinghouses to share information about effective and ineffective program practices 
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In this Roadmap, we more fully develop these ideas. We describe the general principles that should guide a 
government-wide effort to strengthen evaluation functions. We propose broad administrative steps to 
institutionalize evaluation in federal agencies. Finally, we discuss how tlie Executive Branch and Congress 
can jointly make the most effective and efficient use of evaluation. 

General Principles 

"The following general principles should guide efforts to integrate evaluation into program management. 

Scope. Evaluation should he integral to planning, developing, managing, and implementing government 
programs at all stages. Ev'aluation activities should be used to: 

• Make sure that program designs are appropriate to achieve program goals 

• Identify problems during .start-up and correct them before they become entrenched 

• Identify and .share promising approaches that emerge during program implementation 

• Assess the extent to which programs and policies are being vvell implemented 

• To the extent feasible, establish expectations and performance standards at program inception and 
involve stakeholders in refsnittg them as the programs mature 

• Develop appropriate and efficient data collection and reporting system.s and information technology 
support to provide a continuingflow of evaluative information to policy makers and program 
managers 

• E.xamine the extent to which programs roach their Intended beneficiaries 

• Periodically examine selected program features to improve their effectiveness and efficiency 

• Periodically assess program results and service ijuaiity 

• Systematically examine whether an apparently succe.ssful program can be expanded to another 
setting before scaling it up 

Coverage. In general, federal programs and policies should be subject to evaluation. 

Analytic Approaches and Methods. Which anaijffic approaches and methods to use depends on the 
questions addressed, the kind of program evaluated, its implementation ,statu.s, when the evaluation results 
are needed, what they are needed for, and the intended audience. 

No simple answers are available to questions about how well programs work, and no single analytic 
approach or method can decipher the inherent complexities in the program environment and asssess the 
ultimate value of public programs. Furthermore, definitions of ‘'success” may be contested. A range of 
analytic methods is needed, and often several methods — including quantitative and qualitative approaches — 
should be used simultaneously. .Some evaluation approaches are particularly helpful in a program's early 
dev^elopmental .stages, whereas others are mure suited tu ongoing and regularly implemented programs. 

The broader policy and decision-making context also can influence which approach Is most appropriate. 
Sometimes information is needed quickly, requiring studies that can use existing data or rapid data collection 
methods; at other times, more sophisticated long-tenn studies are required to understand fully the dynamics 
of program administration and beneficiary' hehav'iors. 

Over the year.s, the evaluation fieki has developed an extensive array of analytic approache.s and methods 
that can be applied and adapted to a wide variety of programs, dependittg on the program’s characteristics 
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and impiemtniration stage, how the results will be used, and the kinds of decisions that will be made. All 
evaluation metiiods should be context sensitive, culturally relevant, and methodologically sound. Evaluation 
approaches and methods include, but are not limited to: 

• Case studies 

• SurveysS 

• Quasi-expeririiental designs 

• Randomized held experiments 

• Cost-benefit and cost-effectiveness analyses 

• Needs assessments 

• Early implementation reviews 

• Logic models and evaluability assessments 


Resources. Evaluation should be supported through stable, continuous funding sources and through special 
one-time funds for evaluation projects of interest to Executive Branch and congressional policy makers. The 
stable, continuous evaluation funds should be provided through appropriations or program fund set-asides. 
These methods can also be combined to support viable evaluation programs. Pimgram managers should 
authorize and require periodic evaluations of each program throughout its life to provide rich evaluative 
information to policy makers during animal appropriation and cycliccd reauthorization and amendment 
discussions. 

Professional Competence. Evaluations should be performed by prufessionalsS with appropriate training 
and experience for the evaluation activity (such as perfonning a study, planning an evaluation agenda, 
reviewing evaluation results, or performing a statistical analysis). Evaluation is an interdisciplinary field that 
encompasses many areas of expertise. Many evaluators have advanced degrees in, and often work 
coliaburatively with colieagues in allied fields, sudi as economics, political science, applied social research, 
sociology, anthropology, psychology, policy analysis, statistics, and operations research. Federal agencies 
should ensure that the required diversity of disciplines is appropriately represented in internal and 
independent evaluation teams. 

Evaluation Plans. Each federal agency should require its major program components to prepare annual and 
multiyear evaluation plans and to update these plans annually. The planning should take into account the 
need for evaluation results to infom\ program budgeting: reauthorization; agency strategic plan.s: ongoing 
program development and management; and responses to critical issues concerning program effectiveness, 
efficiency, and waste. These plans should include an appropriate mix of short- and long-term studies to 
produce results of appropriate scope and rigor for short- or long-term policy or management decisions. To 
the extent practical, the plans should be developed in consultation with program stakeholders. 

Evaluation quCsStions can spring up unexpectedly and urgently in rCsSponse, for example, to a sudden need for 
information to address a presidential initiative, a management problem, or questions raised by Congress. 
Therefore, evaluation plans should allow for flexibility in scheduling evaluations. 

Dissemination of Evaluation Results. The results of all evaluations related to public accountability should 
be made available publicly and in a timely manner (except where this is inconsistent with the Freedom of 
Information Act or Privacy Act). They .should be easily accessible and usable through the internet. Similarly, 
evaluations of promisSing and effective program practices should be systematically and broadly disseminated 
to potential users in federal agencies. Evaluation data and methods should also— to the extent feasible and 
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with sufficient privacy protections — be made available to professionals anci the public to enable secondary 
analysis and assure transparency. 

Evaluation Policy and Procedures. Each federal agency and its evaluation centers or coordinators 
(discussed betovv) should publish policies and procedures and adopt quality standards to guide evaluations 
within its purview. Such policies and procedures should identify the kinds of ei^aiuations to be performed 
and the criteria and administrative steps for developing evaluation plans and setting priorities, selecting 
evaluation approaches and methods to use, consulting subject matter experts, ensuring evaluation product 
quality, publishing evaluation reports, ensuring independence of the evaluation function, using an 
appropriate mix of staff and outside consultants and contractors, appropriately focusing evaluation designs 
and contracts, and promoting the professional development of evaluation staff. 

Independence. Although the heads of federal agencies and their component organizations should 
parbeipate in establishing evaluation agendas, budgets, schedules, and priorities, the independence of 
evaluators must be maintained with respect to the design, conduct, and results of their evaluation .studies. 

Institutionalizing Evaluation 

Significant progress has been made in establishing evaluation as an integral component of government 
program management. However, additional steps are needed. 

Background. Some federal agencies, such as the Department of Defense, established evaluation offices in the 
late 1950s and early 1960$. Others, including the Departments of Education and of Health and Human 
Services, developed their evaluation functions in the 1970$ within the then-Departnient of Health, Education, 
and Welfare. The authorizing statutes for some of the.se agencies set aside a fixed percentage of program 
funds for evaluation, other departments have added evaluation offices to their organizations although these 
offices have grown and shrutrk over the intervening years. 

One relatively stable evaluation organization has been the Govermnent Accountability Office (GAO), 
previously known as the General Accounting Office. It has remained the largest single government agency 
producing evaluations at Congress's request. 

One of the most enduring evaluation-related functions ha.s been the Govermnent Performance and Results 
Act (GPRA) of 1993. This law encourages each agency to develop an agency-wide strategy and mi.ssion and 
also requires them to determine whether their programs achieve their goals and objectives. GPRA defines 
evaluation as assessing the "manner and extent to which" agencies achieve their goals, tints addressing botlj 
imploniontation and results. In practice, government agencies have implemented GPRA by icsing performance 
indicatuns and measSurement to determine whether they hat^e reached a goal and have conducted few' 
evaluation studies that might shed light on how programs reached their goals, why programs do or do nor 
meet their goals and objectives, and bow to improve programs. As a result, tiie GPRA process produces little 
information to guide programmatic or policy action. 

We propose that government agencies, policy makers, and program managers build on tlie progress already 
made by embracing evaluation as an integral feature of good government. Agencies should consistently u.se 
program evaluation and sy.stematic ana!y.sis to improve program design, implementation, and effectivene.ss 
and to assess what works, what does not work, and why. This comprehensive vision recognizes that 
evaluation is more than simply "looking in the rearview mirror" and needs to be used throughout a 
program’s life as an integral part of managing government prograra-s at all stage.s. 
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For this approach to work, the Executive Branch and Congress wiii need to take action, as described below. 

Executive Branch Role 

As noted earlier, the infrastioicture and practice of program eeaiuatton in federal agencies is somewhat of a 
mixed story. Some agencies have well-developed and .stable evaluation offices; others do not. The .same can 
be said for evaluation funding, scope, policies, planning, and dissemination. 

Different federal agencto-s and programs have different evaluation noed.s, and the maturity and breadtli of 
their evaluation programs vary. In addition, the evaluation funrtioii might be a component of other offices 
focused on such functions as management, planning, research, and policy development, including legislative 
or regulatory development. For example, several agencies have offices of planning and evaluation, research 
and evaluation, or monitoring and evaluation, and some inspectors general have offices of inspections and 
evaluations. In some agencies, the evaluation function is highly centralized or within a large program area; in 
other agencies, the evaluation function is scattered in small offices throughout the agency. 

No single best practice exists for organizing evaluation offices and functions. All oftlie arrangements 
described abtjve have emerged in response to such factors as substantive area, kind of agency, or type of 
evaluation focus. They may' or may not be the mo.st effective models for airrent circumstances. Whatever 
model is chosen, the ev'aluation office must include the functions and possess the attributes described above 
under general principles. 

Based on the general principles discus.sed in the previous section, we propose that agencies in the Executive 
Branch establish one of the following organizational frameworks to stipport evaluatiotr. 

Option 1; Evaluation Centers. Agencies could establish one or more evaluation centers to promote 
evaluation capacity and provide stable oi^anizationa! frameworks for planning, conducting evaluation, or 
procuring evaluation advice or studies from outside organizations. Every program in the agency should be 
assigned to one of the centers for program evaluation. The heads of these evaluation centers should report 
directly to the senior executive of their center’s organizational component. Each of these centers would: 

• Have a stable budget with sufficient funds to plan and carry out an appropriate level of progi'am 
evaluation over several years 

• is.sue policie.s and procedures to guide its evaluation work, including guidance on appropriate 
methods for conducting formative and summative evaluations, as well as developmental evaluations 
to improve evaluative capabilities within agencies. 

• Strategically plan a body of evaiuaticn work for the agency and each agency componentfor which it 
has evaluation responsibility 

• Consu It with agency program and budget offices and, in concert with the agenc/s legislative liaison 
office, with Congress in developing evaluation plan,? 

• Hire profes.siona! evaluators or engage consultants or contractors with the diverse skills necessary to 
plan and execute (or procure) independent evaluation studie,s 

‘ Publksh the results of evaluations related to public accountability of the programs within tlieir 
jurLsdictions 

• Share Information about effective programs and evaluation methods with otlmr government agencies 

• Promote and facilitate the ongoing framing and professional development of the center’s evaluators 
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Option 2: Evaiuation Coordinators. Agencies that choose to distribute their evahiation offices, associating 
small evaiuation offices with individual programs or small groups of programs, should promote evaluation 
capacity and performance by appointing one or more senior officials to: 

• Advise the agency head or senior officials on matters pertaining to evaluation 

• Ensure that each program or program group in the agency has a current annual! evaluation plan. 

• Promote, facilitate, and coordinate the development of evaluation plans for programmatic issues that 
cut across agency lines 

• Facilitate the preparation of evaluation budgets 

• Establish appropriate standards, frameworks, and procedures for evaluation attlvities in the agency 

• Facilitate the development and efficient and effective production of evaiuation plans, designs, 
instruments, and reports by government agency staff or outside eviiiuators 

• Facilitate the dissemination of evaiuation reports related to public accountability 

• Share with otiier agency compoiients information about effective programs and evaiuation methods 

• Promote and facilitate the ongoing training and professional development of evaluators in the agency 

Option 3: Combined Approach. Federal agencies may find it advantageous to use Option 1 and Option 2— 
evaluation centers for large programs, program groupings, and overall evaluation support, and evaluation 
coordinators for distributed evaluation office.s — to ensure the viability of the evaluation function. 

Congress’s Role. The GAO, Congressional Research Serxdce, Congressional Budget Office, Mational 
Academies, and temporary commissions carry out evaiuation and analysis for Congress, usually in fulfillment 
of their oversight role. Congressional committees or subcommittees also conduct some evaluative studies or 
investigations. 

We do not propose to change these organizational structures. Instead, we offer recommendations to 
strengthen the connection between evaluations and the laws that Congress passes. This can be done by 
budding evaiuation expectations into authorizing legislation and explicitly setting aside adequate resources 
for evaluation. 

Authorizing legislation. Program authorization ami periodic reauthorization provide opportunities for 
Congress to establish frameworks for systematic evaluation of new and continuing programs. Congressional 
committees can, through authorizing legislation, provide guidance on or stipulate such activities and 
producLs as: 

• Early implementation reviews to identify start-up problems in such areas as scheduling, contracting, 
and grant maldng and to correct tl-jum before they become mure seriuas 

• Requirements for developing evaiuation plans 

• Evaluation of promising approaches to share among program implementors 

• Development of performance indicators and the means to collect meaningful data on them once the 
program starts 

• Studies reviewing the efficiency of federal program managementas well as tae fidelity of program 
impiemeiitation to the congressional mandate that instituted the program 

• Studies assessing program effects and identifying why programs are or are not effectwe 

• Evaluations of topics of interest to Congress and reports on the results to Congress in support of its 
oversight and appropiiatioirs functioms and to inform futare reauthorizations 

• Establishment, expansion, or amendment of ongoing surveys or otlier data-collection mechanisms to 
become permanent sources of reliable dat:^ 
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• EstahlLshnient: of evaluation centers or evaluation coordinators, as described above 

• Funding for evaluation activities 

Collaboration Between Executive Branch and Congress 

The utUity of evaluation results may be maximized if Congress and tlie Executive Brandi jointly specify broad 
evaluation expectations and concerns in authorizing statutes and appropriations. We recognize that such 
coilaboration wiii not always bo easy or even possibie to achieve. Nevertheless, experience suggests that, 
when possible, a partnership of this kind can help increase the benefits that evaluation provides. 

Looking to the Future 

The U. .‘3. government faces major challenges in the years to come, as well as significant opportunities to 
improve lives, protect the pianet, and create efficiencies. With more thoughtful and more systematic 
integration of evaluation into the responsible planning, management and oversight of programs and the 
application of evaluation results to planning and decision making, the performance of today's programs can 
be improved. Institutionalizing evaluation can also help achieve a more accountable, open, and democratic 
system of governance for future generations. 
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Today more than ever, policymakers need evidence to help inform major decisions about program 
design, implementation, and funding. Whether assessing the likely effectiveness of a new initiative, 
comparing competing approaches to a given problem, figuring out where to cut, or refining a program's 
rules to make it more cost effective, decisions based on rigorous evidence make better use of scarce 
public dollars and improve outcomes for people. 


A Case Study of Eutdence-Based Policymaking. The evolution of fecieral iow-.ncame housing pol'cy 
from its injtifli re'ianceon suop'y-side, piojcct-baied sibsrdies to incmcfe demand-side jubsidies was i 
significantly influenced bv successive rounds of research and evidence. The origins oftoday's housing: 
chclce voucher program can betourc in o 1969 Preslcsem't. Committee oo Housing report, >5 | 
Oncent Ho/nCt whicn reccmmendecl^jfit HUD experiment with cash aibwaices made to families 
according to Their need tongres.? authonzoci jusi such an experiment -n 19/0 Ibis study wasa 
P'OcursortOThc infUenfipl fixpcrimcntoi Mousing Adpwance Progrom {tHAJ}, conducted in the 
19705 ano I'JPOs, in i07i. based on emerging evidence from these experirrenta, President N'xon 
asked Co-igressTO reorient housmg poncy toward p-ovid ngcashessistarce instead of increasing 
production, aased on prelim nary experimemrs. This request !eri to die 197*1 Housing Act that created : 
the Vri.ori Svouf her progmm 

By -.he enriy IQBDi, findmgsfrom three nousing aiiowonte e/fpenmonts {sijmmdrijred in Struyk and 
Bendick 1981) showed that participants enioyed improved QUdiity of housing, that the program did ; 
not cause lent inflation end that vouchers were actuolly cheaper than production programs. Based i 
o”. this ”v:denre, fne Reagan nriminisfrption Impiemnntnd erir^Tinafir smft from suppiy-side 
■j-tei-ventionsforused on construction ot row subsidised housing to de-nqno-sirJe re- tificote and 
voLitner in;ervenUons. All^^ough intore5.t groups on the tr*-'dition«s, loft h*id long beer tfie suiuru hos; i 
supporter^ or^.ovurrirrieut-vu jporleJ cunstructicn (arits previous, Derrsuu'uUc admimstiuLions luid 
expandad construction), the Oc-mocraticCongross appropriated funds for vouchor demonstrations 
from IBSIifo 1987 and authorized the Ri>cg{m admIri'S' ration's p*-eposed vouener program ir- 1988. 

M- the Jidt foilowee, fevKU-tiue ftom inndom control :r als, adiT‘inislr,jtive; ddla, rf'-'d 
inp}ement<>tion research has beer used to st-pngthen the housing voucher program-'i, des-gn any 
adrrjinislraL-or, More sptKificallY, changes :fl how.«'ubsldie5 aro ca!cu!ated. rulos imposed ori 
participating -andlords. allowable rent lovets In orffc-rert types o^neighoo-hoods, as "0 performance 
siandord*; torthe administering local <>genciss have ail peon informed by rigorous ev donee 
cor'duf.ftri over many yfr^rs jNationa: Research Courred 20iJ8) 


Roles for Evidence in the Policy Process 

Often, the conversation about "evidence-based policy" focuses too narrowly on a single question and a 
single step in the policymaking process. In this narrow context, research determines whether a 
particular program "works" or not— whether it achieves the intended outcomes— and that evidence is 
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used as the basis for either funding the program or killing it. In other words, if an initiative or program 
has not been proven effective, it is not "evidence based" and it should not be implemented. 

In reality, policy development occurs in multiple stages and extends over time. New policies emerge in 
response to problems and needs, possible approaches are advanced and debated, new policies are 
adopted and implemented, and 
established policies are critiqued and 
refined, As the figure to the right 
illustrates, evidence can add value at 
every stage, but the questions 
decisionmakers need to answer differ 
from one stage to the next. More 
specifically, policymakers need evidence 
to help them 

• diagnose problems and 
underlying causes; 

• design new policy options and 
assess the likely effects of 
alternatives; 

• demonstrate and evaluate the 
impacts of new, model 
programs; 

• monitor program implementation, measuring costs and performance and their sensitivity to 
different settings; and 

• evaluate the long-term impacts and cost-effectiveness of existing programs. 

No single research too! or methodology can deliver the evidence policymakers need to make informed 
decisions at ail these stages, instead, policymakers and practitioners need a portfolio of rigorous 
research tools to effectively advance evidence-based policy. 

Random Control Trials 

Random control trials— in which people are randomly assigned to participate in a program or serve as 
controls— are often referred to as the "gold standard" for evidence about whether a program is 
effective. And indeed, this approach is extremely powerful because it compares outcomes for a 
program’s participants to the outcomes comparable people achieve without the program. 

To iiiustrate, the Urban Institute is currently evaluating a Youth Alliance initiative that places low-income 
youth in internships to teach them job skills and improve their resumes for college and careers. Often, 
programs of this kind appear effective on the surface, because many participants go on to college or get 
jobs. But what if the young people who were sufficiently motivated and goal-oriented to learn about 
and apply for the special services would have succeeded anyway? To find out whether these programs 
work, outcomes for participating youth must be compared to a control group of equally motivated 



2 



90 


youth, the perfect recipe for a random control trial. In the case of the Youth Alliance initiative, only a 
limited number of internships were available, so the <much bigger) pool of eligible applicants was 
divided by lottery into a treatment group, which was offered internships, and a control group, which 
was not. These two groups of young people are similar in every other respect, including their drive and 
motivation to succeed. So, the evaluation, which is currently under way, will be able to measure the 
impact of the intervention relative to what participants would achieve without it. 

While random control trials constitute the best tool in many circumstances, some programs cannot be 
effectively evaluated in this way. In particular, complex "place-based" interventions (like the new 
Promise Neighborhoods Initiative or the Choice Neighborhoods program] are not good candidates for 
random control trials. These interventions are designed to improve outcomes— educational success, 
health, employment and income— throughout an entire community, not just for individuals who 
participate in a defined program. Part of the approach is to saturate an entire area with new services, 
benefits, or incentives, so that even people who are not directly targeted or enrolled will experience 
spillover effects. These spillovers mean that the statistical framework underlying random assignment 
does not apply. 

In theory, researchers could randomly assign whole communities to receive a place-based intervention 
or not. But place-based programs do not simply implement a simple prescription formulated the same 
way everywhere. Instead, these programs evolve organically in the communities where they are 
implemented and draw different elements from a broad menu of possible services. Each intervention is 
tailored to conditions on the ground. They are aiso continuously improved using data in an ongoing 
development effort that adapts to local circumstances, successes, and failures, with constant feedback 
from outcome data. The combination of spillover effects on people not receiving services, locally 
tailored designs, and continuous improvement mrakes a simple random assignment design the wrong 
choice for evaluating these types of interventions. But alternative methods can produce credible 
estimates of outcomes in the targeted communities compared to what would have happened without 
the intervention (Nichols 2013). 

Other tools constitute the "gold standard" for delivering the evidence policymakers need to answer 
other questions. 

MIcrosImuiation Models 

Often, policymakers want to anticipate the likely effects of policy changes, rather than waiting to 
measure their impacts after the fact. Microsimulation models can forecast outcomes under a wide range 
of "what if" scenarios. Although the development of credible models is complex and time-consuming, 
once a model is in place, it can quickly and efficiently analyze a wide range of alternative policies. 

The Urban-Brookings Tax Policy Center has built a painstakingly detailed model that applies the 
provisions of the tax code to a 270,000-unit sample of "taxpayers" that statistically represents the US tax 
base,^ The model reports the revenue and distributional effects of elements of and changes to the tax 


For a more complete description of the Urban-Brookings tax policy simulation model, see 
http://taxpolicycei5tsr,O'-g/taxtopics/8rief-Oescription-af-tiie-Model-201.3.cfm. 
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code. For example^ recent analyses applied the mode! to questions of equity and efficiency of the home 
mortgage interest deduction, demonstrating that its biggest beneficiaries are households making more 
than 5100,000. This analysis also predicted whose taxes would rise, whose would fall, and how total tax 
revenues would change under several alternatives to the mortgage interest deduction currently under 
discussion (Eng et al. 2013). 

Administrative Data Analysis 

Public agencies at every level of government collect a tremendous volume of information about benefit 
recipients, market transactions, and enforcement actions. These data can be systematicaiiy linked, 
monitored over time, and analyzed to produce retiabie evidence for policymakers. In many cases, 
administrative data provide the building blocks for microsimuiation modeling or the outcome measures 
in a random control triaf. But rigorous analysis of administrative data can also provide answers to 
immediate questions about program design and implementation, 

The Urban Institute is using administrative data to help design streamlined enrollment procedures for 
health insurance coverage under the Affordable Care Act, Publicly funded health insurance programs 
like Medicaid and CHIP require enroiiees to complete vast amounts of paperwork to demonstrate their 
eligibility. This barrier is costly for both applicants and processers, and it prevents some eligible people 
from receiving coverage. With 21st century data and technology, it should be possible to automate 
processes to determine applicant eligibility, improving accuracy and saving public dollars. To test this 
proposition, Urban Institute researchers assembled administrative data from tax returns, state 
workforce eligibility records, and other sources to determine what existing records could verify current 
eligibility for Medicaid. This study will produce potential business rules for initial applications and 
renewal to the Medicaid program that would improve the system's efficiency and reach (Dorn et al. 
2013). 

Performance Measurement 

Performance measurement is a familiar concept throughout the federal government and a key 
element of evidence-based policymaking. The Government Performance and Results Act of 1993 
required federal agencies to report annually on performance indicators beginning in fiscal year 1999. 
Agencies set targets for each performance indicator and report progress against those targets. 

Several federal agencies have responded to the need for data-driven performance reviews by 
developing an approach that consists of regularly held, structured, data-driven performance review 
meetings. This strategic leadership approach, often referred to as PerformanceStat, was initially 
developed by the New York City Police Department in 1994, and has since been adapted for use in other 
local, state, and federal government agencies (Hatry and Davies 2011). 

One of the key challenges of performance measurement is defining indicators that are both meaningful 
and measurable. If agencies focus exclusively on easy-to-collect process measures (like number of 
applications reviewed or service referrals provided), they fail to reflect outputs and outcomes of 
ultimate importance. But if they focus instead on long-term outcomes, they sacrifice near-term 
feedback on program performance. To help policymakers and practitioners develop effective 
performance measures, the Urban Institute, Child Trends, and Social Solutions are collaborating to 


4 



92 


develop PerformWeil, a web portal providing measurement tools and practical knowledge that human 
services professionals can use to manage their programs’ day-to-day performance, information in 
PerformWeil leverages research-based findings that have been synthesized and simplified by experts in 
the field (Urban Institute 2011). 

Quaiitative Research 

Sometimes, fully diagnosing a complex problem, designing an innovative solution, or understanding 
exactly how a program should be implemented requires in-depth, nuanced information that cannot be 
obtained from conventional surveys or administrative data sources. Qualitative research uses in-person 
observation, in-depth interviews, and focus groups to dig deep and explore the behavior of people and 
institutions. Qualitative methods can explain and enrich findings from statistical data, highlight key 
issues that might otherwise be missed, and reveal hypotheses for further testing. 

The Urban institute recently completed an evaluation of the federal New Markets Tax Credit program, 
which has allocated over $3 billion to public-private investments in low-income community 
development. In conjunction with quantitative analyses of administrative and original survey data, 
researchers drew a small random sample of assisted businesses and analyzed each in depth, learning 
about the type of business and the loan it received, the nature of its neighborhood and market, the 
importance of the federal subsidy, and the subsequent economic performance of the business. This in- 
depth information played a critical part in a "triangulation” process that comprehensively addressed a 
complex market intervention (Abravanel et al. forthcoming). 

Qualitative methods can also be invaluable in crafting the specific provisions of a new intervention 
before it is implemented or evaluated on a large scale (and at high cost). For example, behavioral 
economics has taught us that our financial decisionmaking is deeply influenced by subtle contextual 
factors that frame our choices. Programs designed to encourage savings among low-income earners are 
unlikely to succeed unless they take advantage of the "choice architecture" among their intended 
beneficiaries. The StabilityFirst pilot test, conducted in 2010 by Harvard's "ideas42" center on applied 
behavioral economics, enrolled 20 students at Central New Mexico Community College in Albuquerque 
into a prepaid debit card program. The students were interviewed at length both before and after to 
gauge their reactions to the program. A range of issues surfaced, including difficulty resolving customer 
service matters. Participants were reluctant to make calls to the customer service litne, not wanting to 
commit their scarce cellphone minutes to a possibly lengthy call with time spent being transferred or on 
hold {Mills 2011). Information like this enables program administrators to make adjustments that 
correct these "blocking factors," making the subsequent randomized trial far more useful and the 
outcomes sought more likely to come about. 


It almost goes without saying today that policymaking should be evidence based. Scarce public dollars 
should go to programs that target real problems, operate efficiently, and demonstrably achieve their 
intended goals. But policymaking is a messy, iterative process, and the opportunities for evidence to 
inform and strengthen decisions are numerous and varied, instead of relying on a single tool. 
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policymakers and practitioners should draw from a "portfolio" of tools to effectively advance evidence- 
based policy. Using the wrong tool may produce misleading information or fail to answer the questions 
that are most relevant when a decision is being made. Applying the right tool to the policy question at 
hand can inform public debate, help decisionmakers allocate scarce resources more effectively, and 
improve outcomes for people and communities. 
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Mr. Chairman and Members of the Subcommittee: 

I am pleased to submit the following statement to the Subcommittee on Human Resources on behalf of 
ZERO TO THREE, National Center for Infants, Toddlers, and Families. My name is Mattlievv Melmed, 
and I am the Executive Director of ZERO TO THREE, a national non-pi'ofit organization that has worked 
to advance die healthy development of America’s infants and toddlers for over 3o years. I would like to 
start by thanking the Subcommittee for holding this hearing examining what is known about the 
effectiveness of cun'ent programs designed to assist low-income families and individuals and how high- 
quality evidence can best be used to inform the design of social programs at the federal level. Today, I 
want to urge your attention to, and your action on behalf of, a subset of that group who trulv are the most 
vulnerable members of our society: infants and toddlers. 

As this Subcommittee searches for answers to the problem of poverty, I urge you to bear in mind the need 
to start earty' in life through dual generational approaches. Tire effects of early poverty' have a long reach, 
and not taking steps to intervene early with the youngest children — even while working to help their 
parents achieve self-sufficiency - makes later adverse outcomes more likely to occur and more difficult to 
prevent. There is a growing interest nationwide ui early childhood programs in the years immediately 
preceding kindergarten. It is important to note that for our most vulnerable at-risk infants and toddlers, the 
achievement gap often emerges long before they reach the preschool door. We know that high quality 
early learning experiences during the infant and toddler years are associated with attributes important to 
later school success, including early competence in language and cognitive development, cooperation 
with adults, and the ability to initiate and su.stain positive exchanges with peers. Focusing policy 
responses on very young children and their families could yield positive results, especially in the long 
tenn. 

Introduction 

Our nation’s infants and toddlers are the group most likely to be living in distressed economic conditions. 
Nationally, almost half (48%) of children under age 3 live in low-income families, including 25% that 
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live in families in poverty. These statistics are particularly disturbing, because research shows that tire 
riming of economic hardship in a child's life matters; poverty experienced at such a young age is 
particularly harmful both for short term development and outcomes later in life, including in education 
levels, social and emotional health, and physical health. 

During the first 3 years of life, tire brain undergoes dramatic development as the child acquires the ability 
to think, speak, learn, and reason. A baby's early experiences shape the brain’s architecture into a 
foundation for learning, health, and eventual success in tire workplace. Family resources, from income to 
parental education to environmental comforts, play a role in these eruly experiences and thus the 
developmental outcomes. 

Poverty often leads lo multiple risk factors. Children with risk factors such as living in low-income 
households, abuse or neglect, prenatal exposure to alcohol or other substances, and low' parental 
education, have a higher incidence of developmental delays and disabilities than the general population. 
Disparities emerge as early as 9 months and widen by 24 mouths of age.‘ Infants and toddlers from low- 
income families are less likely to be in very good health or receive positive behavior ratings than those 
from higher income families. By age 2, toddlers in the lowest socioeconomic quintile are behind all other 
children in measures of cognitive skills and emotional attachment.” 

These adx-erse early experiences can weaken babies ’ brain development and follow them their entire 
Ifves. placing them at greater risk for later school failure and health problems as adults. When babies and 
toddlers experience chronic deprivation and .stress there are costs in lost opporliinity and achievement 
deficits to society and our economy unless we interx’ene. 

Early delays in development that are not detected lead to widening gaps that put children on the path to 
poor outcomes, and have costly repercussions. Delays in speech and physical development as an infant 
can be signs of later learnuig disabilities, autism, or cerebral palsy.”' One in six children will experience a 
developmental disability or behavioral problem before age 1 8, but fewer than half of those problems are 
detected before school entry.”' Without increased investments focused on the availability and accessibility 
of quality early care and education experiences, many infants and toddlers ivill continue to be left behind. 
On the other hand, with high quality, effective services, those infants and toddlers who are at-risk for 
compromised development will be better equipped to reach their full potential in life. 

IlVTERV'ExMNG EaRLY 

The brain is most flexible, or 'plastic,’ eaiiy m life to accommodate a wide range of environments and 
interactions, and can be rewired in reaction to significant changes in their lives. Early plasticity points to 
the importance of early intervention; it is easier and more effective to influence a young child’s 
developing brain architecture than to offer remedial programs later in life.' Reaching children well before 
they enter school can strengthen their chance.s for later success, despite the poor life conditions they face. 

Research confirms that the early years present an xmparalleled window’’ of oppoitunity’ to effectively 
intervene with at-risk children.'' To be effective, inten’entions must begin early and be designed with the 
characteristics and cxpcrieuce.s of these infants, toddlers, and families in mind.''^ Intervening in the early 
years can lead to significant cost .savings over time through reductions in child abuse and neglect, 
criminal behavior, welfare dependence, and substance abuse. If services are not provided until a child is 6, 
7, or 8 years of age, the most critical opportunity' for prevention and intervention is missed.'’'” 

Given this early window of opportunity, there are a number of ways that policvmiakei's and practitioners 
can intervene to improve outcomes for infants and toddlers. Proven approaches — some beginning during 
the important prenatal period — can help diminish the gaps and promote stronger social-emotional 
foundations. Economists estimate that for every dollar invested in early childhood programs, savings of 
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S3. 78 to Si7.07 can be expected ui future public expenditures, in part by reducmg the occurrence of 
negative educational and life outcomes. 

Effectivt Programs 

When focusing on specific methods of delivering ser\'ices, it is important to think m terms of developing 
a comprehensive svstem of services that provide a prenatal through pre -kindergarten continuum. Such a 
system would ensure that the critical needs of vulnerable infants and toddlers — regardless of the setting in 
which they might be reached — are included in early childhood planning. That system would help parents 
and early childhood professionals promote healthy development across all domains. 

Programs and seiv'ices in this system should have an important element of supporting parents in forging 
bonds with tiieir children since developing strong attachments provide the needed foundation for a child 
to explore and learn as well as to regulate their emotions as tliey interact with othei's (social and emotional 
development). Such services should also help parents and babies engage in play, reading, and other 
activities that foster early language skills (cognitive development) and they should promote good nutrition 
and attention to well-child care (physical development). 

Model early childhood programs that deliver carefully designed interventions with well-defined 
objectives and that include well-designed evaluations have been shown to influence the developmental 
trajectories of children whose life course is threatened bv socioeconomic disadvantage, family disruption, 
and diagnosed disabilities. Programs that combine child-focused educational activities with explicit 
attention to parent-child interaction patterns and relationship-building appear to have the greatest impacts. 
In contrast, services that are based on generic family support, often without a clear delineation of 
intervention strategies matched directly to measurable objectives, and that are funded by more modest 
budgets, appear to be less effective.^ 

Effective early clnldliood programs also impact parents’ efforts to become economically .self-sufficient. 
Effects range from enabling parents to work to support their families to promoting education and training 
that can better position parents to move into the job market. 

Some examples of proven programs mclude; 

• Early Head Start 

Early Head Start is the only federal program specifically designed to ensure that ail yoimg 
children have the same oppoiumities by improving the early education experiences of iow- 
mcome infants and toddlers. Early Head Starr offers opportunities for early learning experiences, 
parent support, home visitation, and access to medical, mental health, and early intervention 
services. 

The Congressionaiiy-mandated Early Head Start Research and Evaluation Project - a rigorous, 
large-scale, random-assignment evaluation - concluded that Early Head Stan is making a positive 
difference in areas associated with children’s success in school, family self-stifficiency, and 
parental support of child development. For example. Early Head Start produced statistically 
significant, positive impacts on standardized measures of children’s cognitive and language 
development.^' Impacts were not limited to children. The progi'am had significant positive 
impacts on participation m education and training among parents with some impacts on 
employment beginning to emerge late in the study."'’" Findings include: 

o Statistically significant, positive impacts on standardized measures of cognitive and 
language development, 
o More positive approaches to learning, 
o Fewxr behavior problems. 
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o Reductions in the negative impacts of parent stress and risk factors on child language 
and self-regulator)’ development. 

o Parents were more involved and provided more support for learning, 
o Parents had reduced risk of dejiression. 
o Positive impact on child-father interactions. 

Studies also sliow that children who participated in Early Head Start had significaiith' larger 
vocabularies and scored higher on standardized measures of cognitive development than children 
in a control group who did not participate in Early Head Start. Additionally,. Early Head Start 
children and parents had more positive interactions,, and these parents provided more si^port for 
learning than did those in a control group.““ 

Despite the program’s proven ability to lessen the negative effects of poi^erty', consistently low 
levels of federal funding and increasing child poverty have kept the program's capacity low. In 
FY2012, less than 4 percent of eligible children are served by EHS initiatives.^' 

Child Care 

Most people think of child care as a service that enables parents to work — and it certainly is a 
vital piece of the puzzle for families seeking economic security for their young children. Yet, it 
also has important implications for early brain development. Second only to the immediate 
family, child care is the setting in which early childhood development unfolds for 6 million 
infants and toddlers who spend some part of their day in iion-parental care. Child care that is of 
poor quality can have a detrimental effect ou early development. And the children who could 
benefit most — those from low-income families — are the ones most likely to be in poorer quality 
care. High quality care for very young children is scarce and out of reach for many families. 
Nationally, the cost of an infanf s child care for single mothers ranges from 25% to 69% of the 
mother’s median income, and the cost for married couples ranges from 7% to 16% of their 
median income.’^'’ 

High quality child care i.s associated witli outcomes that all parents want to see in their children, 
from cooperation with adults to the ability to initiate and sustain positive exchanges with peers, to 
early competence m math and readmg — all key ingredients to later school success. Research 
indicates that the strongest effects of quality care are found with at-risk children — children from 
families with tew resources and under great stress. 

Specifically, studies that examine children's development over time have shown dial higher 
quality child care is a predictor of improvement in children's ability to understand .spoken 
language, communication skills, verbal IQ skills, cognitive skills, behavioral skills, and 
attainment of higher math and language scores — all of which impact later school success.’^" 
Research also indicates that participants in high quality^ child care and early education progi'ams 
may also experience lower levels of grade retention and placement in special education 
classrooms.^^’ 

One of the features tliat distinguish higher quality' care is the amount of language stimulation 
provided. High quality child care, where provider,s are both supportive and offer more verbal 
stimulation, creates an environment w'here chiidi'en are likelv to show advanced cognitive and 
language development.^'"” For virtually ever}' developmental outcome that has been a,ssessed. 
quality of care also show.s positive associations with early social and emotional development.^'^ 
Higher quality' care is generally related to more competent peer relationships during early 
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cliildhood and iiito the school years. It provides environments and opportunities for socialization, 
problem-solving, empathy building, sharing, and relating. 

Evidence-Based Home V’^isiting 

Home visiting has been demonstrated to be an effective method of supporting families as they 
guide their children’s development, particularly as part of a comprehensive and coordinated 
system of services. These voluniarv' programs tailor .services to meet the needs of individual 
families and offer information, guidance, and support directly in the home environment. While 
home visiting programs vary in goals and content of services, in general, they combine parenting 
and health care education, child abuse prevention, and early intervention and education services 
for voung children and their families. Home visiting is a means to establish trusting relationships 
widi families and deliver or link them to necessary resources and supports. Depending on the 
model used services may include healtli, parent education, family support, and other services to 
promote maternal well-being and family self-sufficiency. 

The U.S. Department of Health and Human Services and Mathematica Research have identified 
1 2 evidence-based home visiting models that have demonstrated positive impacts in a set of eight 
domains. Eleven of these are designed to include infants and toddlers and their families; Child 
FIRST, Early Head Start-Home Visiting, Eariv Intervention Program for Adolescent Mothers 
(EIP), Early Start (New Zealand), Family Check-Up, Heaitliy Families America (HFA), Healthy 
Steps. Nurse Family Partnership (NFP), Oklalioma’s Community-Based Family Resource and 
Support, Parents as Teachers (PAT), and Play and L.eaming Strategies (PALS) Infant. 

Depending on the model used, positiv'e impacts have been shown in one or more domains, 
including child health, cliild development and school readiness, maternal health, reductions in 
child maltreatment, family economic self-sufficiency, positive parenting practices, linkages and 
referrals, and reductions in juvenile delinquency, family violence, and crimc.“ In randomized 
trials, home visiting programs were found to be effective methods for delivering these essential 
parent support services. When compared to control group counterparts, parents with very low 
incomes who participated in a home visiting program were more likely to read aloud, tell stories, 
say nursery rhymes, and sing with their child.™' Participants in home visiting programs also 
created more developmentally stimulating home environments™", had more responsive 
interactions with their children™'", and knew more about child development.™'" 

Home visitation programs can counteract the negative coii.sequeiices of economic insecurit}' and 
encourage success not only at home but also m school and at work. Home visitation programs 
help parents enroll in edticatioual and training programs and pursue employment opportunities. In 
a serie.s of randomized controlled trials of a nurse home visitation program serving unmarried 
low-income women, S2% more participants worked compared to the control group in the period 
up until their child turned 4.™' In another trial of the same program, participants were twice as 
likely to be employed as tire control group at their child’s second birthday.™” A randomized 
controlled trial of another program demonstrated high paiticipation in school or training 
compared to the rate of the control group ; a particular benefit of this program was the setting of 
concrete goals with the mothers for their education and professional development.™”'' Finally, a 5- 
year foiiow-up study of another home visitation program found higher monthly income for smdy 
participants.™'’"’ 


Conclusion 

During the first three years of life, chiidi'en rapidly develop foundational capabilities — physical, social- 
emotional, and cognitive — on which subsequent development builds. These areas of development are 
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inextricably related. Yet, too often, we ignore the early years of a child’s life in making public policy, 
waiting until at-risk children are already behind physically, emotionally, or cognitively before significant 
investments are made to address their needs. We must change this pattern and invest in at-risk infants and 
Toddlers early on, when that investment can have the biggest payoff — ^preventing problems or delays that 
become more costly to address as the children grow older. 

All young children should be gi\'en the opportunity to succeed in school and in life. Ensuring that infants 
and toddlers have strong families who are able to support their healthy development w'ill help lay the 
foimdation for a lifetime of success. Wc must increase federal investments so that infants, toddlers and 
their families have acce,ss to developmentally appropriate early learning programs such a.s Early Head 
Start, high quality and affordable child care, and home visiting services to help ensure that they are ready 
for school. 
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Capital IDEA Board of Directors 


Testimony of Paul Skeith 
Chair, Capita! iDEA Board of Directors 
Leader and Strategy Team Member, Austin Interfaith 
Managing Partner, Richards, Rodriguez, and Skeith 


Thank you for the opportunity to submit this written testimony about how Capita! IDEA lifts working 
poor adults from poverty into good careers through a community college education. We are very proud 
of our 1,000+ formerly working poor who have started new careers as nurses, network administrators, 
energy techs, and other skilled workers needed by our economy. As you heard in testimony from Tara 
Smith of the University of Texas' Ray Marshaii Center, rigorous evaluations demonstrate that Capital 
iDEA graduates nearly triple their earnings and deliver a 501 percent return-on-investment to taxpayers. 
Over 1,000 working poor adults who formerly needed taxpayer assistance are now substantial taxpayers 
themselves. Moreover, they serve as role models for their children, 90 percent of whom graduate from 
high school and 75 percent go on to higher education— permanently breaking the cycle of poverty. 

The strategy that the Texas Industrial Areas Foundation Network of Organizations first developed with 
Project QUEST in San Antonio in the 1990$ is simultaneously complex and simple. The most important 
ingredient of our success is a mutual commitment between the student and the community. Yes, we 
invest substantia! money and time in our students: our sectoral workforce intermediary organizations 
like Capital IDEA pay for (or leverage with our partners) tuition, fees, books, vaccinations, tools, exam 
fees, uniforms, child care, bus passes or gas money, credentials, and everything else short of bare living 
expenses. But there is much more to success in a community college than just resources. 

First, people find out about us from institutions they trust - the congregations of Austin Interfaith and 
our sister Industrial Areas Foundation organizations, friends and family, and social service agencies. 

They know that these institutions believe in them, care about them, and also expect accountability from 
them. 

Second, they go through a six-week "admissions" process. The first step is to ask them to explore the 
high-demand, high-skili careers our employer partner's offer. Next we measure their baseline math and 
reading skills and assess their fit for their career of Interest. Then together we develop a three or four- 
year financial and educational plan that will raise them from as tow as fifth-grade math and reading skills 
to a community college graduate. The final and most important step is a heart-to-heart conversation 
about commitment to the plan and, once graduated, to give back to the community that supported and 
believed in them. We don't often say "No," but we do say "Not yet," and help them identify the things 
needed to get ready. 

Third, once they begin, students team with a group of their peers and a Career Navigator, who is a 
student's guardian angel throughout their journey. Students meet weekly as a group with their 
Navigator and his or her responsibility is to get them successfully through their education and into a 
career, no matter what, if someone's spouse or partner becomes abusive, we help them get into a 
shelter. If the shelter is closed, we take our credit card and get them into a hotel until the shelter opens. 
If someone's child acts up in school, we help them deal with the school system. If someone has an 
undiagnosed learning disability, we help them get support. 
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Currently, the Federal funding for these efforts is limited and short-term. We leverage everything 
available from the Workforce investment Act, federal child care dollars. Temporary Assistance for Needy 
Families, and other sources. However, none will support ai! of a student's needs throughout this 
multiple year and muitiple step process. From the beginning the Industriai Areas Foundation 
organizations and their business community partners develop iocai general revenue and philanthropic 
support to fill the gaps left by traditional workforce development funding. We have also developed a 
state program, the Innovative Adult Career Education program, which we offer as a mode! for future 
Federal efforts. 

We emphasize that resources are not enough on their own. Legislation needs to fund the full-on case 
management and counseiing/advising of Career Navigators for non-traditional adults and other first- 
generation college students, if one does not have a college-educated parent to guide him or her 
through the college maze, he or she needs someone tike our Career Navigator to play a similar role. 

industrial Areas Foundation organizations have put this strategy to work effectively in cities and regions 
across Texas and in Arizona, Louisiana, and Iowa. We firmly believe that the working poor are not a 
liability for our society but our biggest opportunity. No students wili work harder than parents who 
have determined to change their lives and their children's lives. We have had women give birth on a 
Thursday and then take mid-terms the next week. The time and resources necessary to educate an 
adult may seem like a lot from the perspective of traditional workforce development policy, but it is a 
very little from the perspective of a lifelong career contributing to our economy and tax base. 

Thank you. 
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PEW Charitable Trusts 

Statement for the Record from the Pew Charitable Trusts 
U.5. House Committee on Ways and Means, Subcommittee on Human Resources 
Hearing on Evaluating Efforts to Help Families Support their Children and Escape Poverty 

Thank you for the opportunity to share a statement for the record with the House Ways and Means 
Subcommittee on Human Resources as part of the July 17, 2013, record on Evaluating Efforts to Help 
Families Support Their Children and Escape Poverty. We applaud the Subcommittee for their leadership 
and share your commitment to ensuring that states are investing in what works. 

The Pew Charitable Trusts (Pew) is a global research and public policy organization, operating as an 
independent, non-partisan, non-governmentai organization dedicated to serving the public. Our work 
lays the foundation for effective policy solutions by informing and engaging citizens, linking diverse 
interests to pursue common cause and insisting on tangible results. Our projects encourage efficient, 
responsive governments - at the local, state, national and international levels - serving the best 
interests of the people. 

in his opening statement. Chairman Reichert noted that, "Congress and the Administration should fund 
what works so we can deliver better results to those in need." We agree and strongly encourage 
members of the Subcommittee and Congress more broadly, to support reauthorization of the Maternal, 
infant and Early Childhood Home Visiting (MIECHV) program. This program, as was noted during the 
hearing, is uniquely designed and highly-effective in delivering improved outcomes for children and their 
families. This statement for the record provides a summary of recently released. Pew-commissioned 
research on home visiting and an overview of our partnership with states as they work to strengthen 
their home visiting systems. Both reinforce the call for reauthorization of MIECHV. 

New home visiting research deepens understanding of how to achieve success 

Voluntary, home visiting is an effective, evidence-based and cost-efficient way to ensure that children 
have the opportunity to grow up healthy, ready to learn and able to become productive members of 
society. Importantly, high quality programs can also yield improved educational and employment 
outcomes for mothers, equipping families with needed tools to move beyond poverty.^ Well-designed 
and well-implemented programs have been shown to improve the lives and future prospects of children 
and families and to yield positive returns on taxpayer investments. ^ 


^ LeCroy, C. W., & Krysik, J. {2011). Randomized trial of the Healthy Families Arizona home visiting program. Children and Vouth 
Services Review, 33(10): 1761-1766. 

^ DuMont, K., K. Kirkland, S. Mitchelt-Herzfeld, S. Ehrhard-Dietzel, M. Rodriguez, E. Lee, C. Layne, and R. Greene, Final Report: A 
Rondomized Trial of Healthy Fotvilies New York (HFNY): Does Home Visiting Prevent Child Maltreatment? 2010. As of February 
2011:http://www.ncjrs.goy/j>ciff:'lo.sl/ni|/gr3nts/132945.pdf and Karoly, LA, Kiiburn, MR, and Cannon, JS. Lynn A., M. 
Rebecca Kiiburn, and Jill S. Cannon, Early Childhood Interventions: Proven Results, Future Promise (Santa Monica, 
CA; The RAND Corporation, 2005). 
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Neuroscientists tell us that our experiences as infants and toddlers greatly affect later life. The years up 
to age three are a unique and critical developmental period in brain growth. Researchers have found 
that our parents' actions (or inaction), and the environment they provide, shape the brain's 
fundamental capacity during this "once in a lifetime" developmental period. In other words, our 
potential isn't something we're just born with, it's also built up by what our parents do— the way they 
interact with us, speak with us, nurture and play with us. Sharing knowledge like this and, more 
Importantly, teaching parents how they can apply it while raising their child, is an essential part of home 
visiting. 

For families facing difficult situations, the information and support home visitors provide have proven to 
Improve outcomes for children and parents. Decades of research on home visiting have shown 
Important gains in school achievement and health and large decreases in rates of low birth-weight 
babies, infant mortality, and involvement with the criminal justice system. 

These outcomes matter, for families, for communities and for taxpayers. 

As part of our commitment to advance quality home visiting across the country, the Pew Charitable 
Trusts, with generous support from the Doris Duke Charitable Foundation and the Children's Services 
Council of Palm Beach County, sponsored independent research to build the evidence needed to inform 
policymakers' decisions, and to advance effective practice in home visiting programs. 

This new research points to the importance of program quality and target population—and the 
interactions between them— in determining ultimate outcomes for children and families. The critical 
question has become not just "What works?" but "What works for whom under what circumstances?" 

The Pew-commissioned research begins to answer this key question, provide new mechanisms by which 
states can evaluate the effectiveness and appropriateness of their home visiting investments, and 
identify needed improvements. 

Key findings from our research: 

> Being truly evidence-based is an ongoing process that goes beyond model selection to include 
continuous data monitoring, analysis, feedback, experimentation, and testing in order to 
improve quality and maximize outcomes for children and families. 

> Programs need to maintain a strong focus on relevant content areas (such as parenting skills and 
children's cognitive development) in order to achieve positive outcomes for children and 
families. 

Within the context of individual studies, researchers also found: 

> In addition to other benefits, high-quality home visiting can significantly improve first-graders' 
school readiness and reduce the rate at which they repeat first grade. 

> At-risk mothers who already have children can benefit from home visiting as much as first-time 
mothers can. 

Pew's partnership with states provides critical lessons in "what works" for improving outcomes for at- 
risk families and children 
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Pew applauds Congress for recognizing the value of investing in voluntary, evidence-based home visiting 
through MiECHV. importantiy, the legislation struck a thoughtfui balance - allowing for tailoring and 
innovation at the state level and providing accountability and technical support at the federal level - 
that has yielded a strong state-federal partnership. MiECHV provides a critical framework of 
accountability for ensuring that the federal investment truly yields improved outcomes for children and 
families. This same kind of framework is needed to bolster the investments states have long been 
making in home visiting to ensure that there is one coordinated system of home visiting operating within 
a state. Building one system will help ensure that program quality, effectiveness and accountability are 
consistent, regardless of funding stream. 

For that reason, and informed by the growing body of implementation research including that which is 
summarized above, Pew has been partnering with states to enact legislation that builds upon MiECHV 
and better positions states to more effectively and efficiently leverage the array of funding streams. Ten 
states have enacted legislation that buiids upon the MIECHV legislation; Arkansas, Connecticut, Iowa, 
Kentucky, Maryland, Michigan, New Mexico, Ohio, Texas, and Vermont. 

in general, the legislation does the following three things: 

First, it clearly defines the purpose and expected outcomes of the state's home visiting investment. This 
is vital to ensuring that programs target the families and work toward the outcomes that are the state's 
highest priorities, it is also key to creating a system that fairly measures the performance of ail 
programs. 

Next, the legislation invests taxpayer dollars in home visiting programs that have a proven record of 
success. Once a state knows which families it wants to serve and the outcomes it seeks, poiicy makers 
should took for programs with solid evidence to meet those objectives. 

And finally, the state's responsibility does not end at reviewing the evidence and picking the programs 
to fund. As Congress recognized in the MIEHCV program, ensuring effectiveness means states must track 
public dollars and outcomes across all programs, even those that have demonstrated solid results in the 
past. States are working to establish that same infrastructure for their home visiting system more 
broadly to ensure that the programs they are funding actually "move the needle" on the goals the/ve 
set. For example, are they reducing infant mortality or resulting in kids being more ready for school? 

MIECHV also encourages states to invest 25 percent of funds in promising programs that are still being 
evaluated. Pew strongly supports this because it aiiows the state to promote Innovations that may help 
families for whom programs with stronger evidence haven't worked or been tested. Additionally, a few 
states have innovative long standing programs that are ready to be rigorously evaluated. And, 
importantiy, funding innovation may identify ways the state can achieve results comparable to more 
established programs at less cost. 


Pew shares your commitment to investing in what works along with ongoing monitoring and 
measurement of the effectiveness of programs. Voluntary, evidence-based home visiting is proven to 
help families support their children and escape poverty, in addition to the real and lasting impact these 
programs have on the families they serve, research shows that these investments can yield returns on 
taxpayer dollars. We believe it is critical to continue learning from the research and for the federal 
government to continue encouraging states to improve the systems that monitor program outcomes. 

To that end, we note the looming need for reauthorization of the MIECHV program. As noted by Jon 
Baron, President of the Coalition for Evidence-Based Policy, in his testimony before the Subcommittee, 
the MIECHV program exemplifies the kind of smart investment that yields strong returns. We encourage 
members of the House of Representatives to support reauthorization of this highly-effective program. 
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Chautauqua Opportunities 


Te^mpny ; yVays and Means Committee 
HpwTpday's Fraarnent^vyelfereSystemFfflJs.tp.LKypPoor.FamjBes 


Chautauqua Opportunities Inc. has been an anti poverty agency since 1965. For the past 15 
years we have restructured our service model to be integrated and comprehensive. We are an 
evidence based agency and we have created a service model dial is asset focused and 
provides individuals in poverty a ladder out . We measure individual and household progress 
toward self sufficiency and economic security against continuums of care based on ROMA 
categories (Crisis to Thriving). It is a customer driven model that is strength based and utilizes a 
wide array of services as “toots’ to achieve short and long range transformative life goals. We 
deliver our services from a cognitive behavioral approach and have full supports in place that 
reinforce behavioral change theory. 


From our experience we make the following recommendations: to remove existing barriers and 
to create better opportunities to assist families working their way out of poverty. 


• Current Welfare system needs to be re-designed to reflect financial life after welfare (copays in 
line with overall percentages of net cash) 

• There needs to be a planned transition off welfare ( assuming the recipient will be attaining a 
relatively low paying job)....supportive services should provide the necessary supports at levels 
that allow slightly more cash in hand while working (comprehensive enrollment that matches 
need) 

• Soft employment skill development should continue - post welfare - to increase earning 
capacity 

• Supportive services ^ould be designed to incrementally decrease as wages increase 

• Basic service design; (applicable to employable individuals - not to specific populations that are 
not able to be self supporting) 

• No service should be “for life” and do not force divesting of assets to qualify 

• Do not incentivize default and bad credit in order to become eligible for greater benefit. 

• Address waiting lists for services that are critit^i for employment (child care, housing, health 
care, transportation, food) 

• Structure services to support asset acquisition and a ladder out of poverty - 
« All services should support financial education and financial repair 

New areas for federal concern: 

• Promote integration of services; 

0 Centralize oversight of human service funding... (result; lower gov’t audit cost, alt 
resources would support similar goals, would bring coordinated service delivery at local level 
to customers - resulting in better outcomes and lower costs) 

o Mandate that RFPs reflect integration, holistic customer response, outcome tracking, 
centralized services within organization 

o Encourage organizations to develop economies of scale and reward good 

management(performance based contracting - rather than grant funding) 

• Mandate that states' demonstrate streamlined contract process across 
beaurocracies 

o Mandate centralized intake and customer file by all organizations responding to federal 
RFPs 

o Promote administrative model that capitalizes on volume and saves money and is more 
efficient and effective 

• Expect that agencies move to electronic files (efficient, cost elTective, able to be 
audited off site, better off site back up for disaster recovery - eliminates agency 
redundancy in staff and files) 

• Promote asset focused poiicies that benefit low and moderate income Americans 
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History with local stakeholders/ comprehensive sector involvement: leverage Federal, state, 
local and private support, create local opportunities, breakdown local barriers 
Grassroots connections: involve low income residents in creating solutions and building their 
future 

Comprehensive service delivery mechanism: blending service options expands services and 
reduces cost while increasing Impact on outcomes 

Utilizing all dollars to serve multi level purposes simultaneously: Individuals and families, towns 
and municipalities, institutions (ex: schools, health care, housing, economic development) 

o 



